Driver assistance apparatus and control method for the same

ABSTRACT

Disclosed are a driver assistance apparatus and a control method for the same. The driver assistance apparatus includes at least one camera configured to generate a main image by capturing an image of a periphery of a vehicle, a communication unit configured to receive a plurality of sub images generated by at least one other vehicle, and a processor configured to: select at least one of the sub images based on a predetermined condition or user input; and to generate an expanded image using the main image and the selected sub image.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the priority benefit of Korean PatentApplication No. 10-2015-0081497, filed on Jun. 9, 2015 in the KoreanIntellectual Property Office, the disclosure of which is incorporatedherein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a driver assistance apparatus and acontrol method for the same and, more particularly, to a driverassistance apparatus which provides a vehicle driver with images ofblind spot areas and a control method for the same.

2. Description of the Related Art

A vehicle is an apparatus that transports, for example, people or cargofrom one place to another place via driving of wheels. Examples ofvehicles include two-wheeled cars such as motorcycles, four-wheeled carssuch as sedans, and trains.

In recent years, in order to increase the safety and convenience of auser who uses the vehicle, technology to equip vehicles with, forexample, a variety of sensors and electronic devices is beingaggressively developed. In particular, for example, various apparatusesfor user driving convenience are being developed.

Among these, an Around-view Monitoring (AVM) system is configured togenerate a plurality of images by capturing an image of the periphery ofa vehicle over 360 degrees using a plurality of cameras and to composethe generated images, thereby displaying a so-called around-view screenthat seems to capture an image of the vehicle from above.

The driver can receive assistance operating the vehicle when viewing thearound-view screen. However, due to the fact that the space that can beincluded in the around-view screen is very limited, the utilization ofthe around-view screen is limited to a few situations such as, forexample, parking or slow speed driving (below approx. 20 km/h).

Therefore, there is a requirement for technologies to provide a vehiclewith an image of a wider area than the area, an image of which can becaptured by cameras mounted to the corresponding vehicle, through theuse of images generated by other vehicles.

SUMMARY OF THE INVENTION

Therefore, the present invention is made to solve the problems asdescribed above and an object of the present invention is to provide adriver assistance apparatus which generates an expanded image usingdifferent traveling images generated by other vehicles, therebyproviding an image of an area which cannot be captured by camerasmounted to a vehicle occupied by a driver and a control method for thesame.

Objects of the present invention should not be limited to theaforementioned object and other not-mentioned objects will be clearlyunderstood by those skilled in the art from the following description.

In accordance with one embodiment of the present invention, the aboveand other objects can be accomplished by the provision of a driverassistance apparatus including at least one camera configured togenerate a main image by capturing an image of a periphery of a vehicle,a communication unit configured to receive a plurality of sub imagesgenerated by at least one other vehicle, and a processor configured toselect at least one of the sub images based on a predetermined conditionor user input and to generate an expanded image using the main image andthe selected sub image. Details of other embodiments are included in thefollowing description and the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentinvention will be more clearly understood from the following detaileddescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a view illustrating an outer appearance of a vehicle providedwith a driver assistance apparatus according to an embodiment of thepresent invention;

FIGS. 2A to 2C are views referenced to explain cameras attached to thevehicle of FIG. 1 according to the embodiment of the present invention;

FIGS. 3A to 3C are various exemplary internal block diagramsillustrating the driver assistance apparatus according to variousembodiments of the present invention;

FIGS. 4A and 4B are various exemplary internal block diagramsillustrating a processor of FIGS. 3A and 3B;

FIGS. 5A and 5B are views referenced to explain operation of theprocessor of FIGS. 4A and 4B;

FIGS. 6A and 6B are views referenced to explain operation of the driverassistance apparatus of FIGS. 3A to 3C;

FIG. 7 is an internal block diagram illustrating one example of thevehicle of FIG. 1;

FIG. 8 is a flowchart illustrating a control method of the driverassistance apparatus according to one embodiment of the presentinvention;

FIGS. 9A to 9D are views explaining the operation of the driverassistance apparatus according to one embodiment of the presentinvention;

FIGS. 10A to 10D are views explaining the operation of the driverassistance apparatus according to one embodiment of the presentinvention;

FIGS. 11A to 11D are views explaining the operation of the driverassistance apparatus according to one embodiment of the presentinvention;

FIGS. 12A to 12D are views explaining the operation of the driverassistance apparatus according to one embodiment of the presentinvention;

FIGS. 13A to 13E are views explaining the operation of the driverassistance apparatus according to one embodiment of the presentinvention;

FIGS. 14A to 14E are views explaining the operation of the driverassistance apparatus according to one embodiment of the presentinvention;

FIGS. 15A to 15C are views explaining the operation of the driverassistance apparatus according to one embodiment of the presentinvention;

FIGS. 16A to 16D are views explaining the operation of the driverassistance apparatus according to one embodiment of the presentinvention;

FIGS. 17A and 17B are views explaining the operation of the driverassistance apparatus according to one embodiment of the presentinvention;

FIGS. 18A and 18B are views explaining the operation of the driverassistance apparatus according to one embodiment of the presentinvention;

FIG. 19 is a view explaining the operation of the driver assistanceapparatus according to one embodiment of the present invention;

FIGS. 20A and 20B are views explaining the operation of the driverassistance apparatus according to one embodiment of the presentinvention;

FIG. 21 is a view explaining the operation of the driver assistanceapparatus according to one embodiment of the present invention;

FIGS. 22A and 22B are views explaining the operation of the driverassistance apparatus to control the display state of an expanded imageaccording to one embodiment of the present invention; and

FIG. 23 is a flowchart illustrating a control method of the driverassistance apparatus according to one embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, the embodiments disclosed in the present specification willbe described in detail with reference to the accompanying drawings, andthe same or similar elements are denoted by the same reference numeralseven though they are depicted in different drawings and redundantdescriptions thereof will be omitted. In the following description, withrespect to constituent elements used in the following description,suffixes “module” and “unit” are given or mingled with each other onlyin consideration of ease in the preparation of the specification, and donot have or serve as different meanings. Accordingly, the suffixes“module” and “unit” may be mingled with each other. In addition, in thefollowing description of the embodiments disclosed in the presentspecification, a detailed description of known functions andconfigurations incorporated herein will be omitted when it may make thesubject matter of the embodiments disclosed in the present specificationrather unclear. In addition, the accompanying drawings are provided onlyfor a better understanding of the embodiments disclosed in the presentspecification and are not intended to limit technical ideas disclosed inthe present specification. Therefore, it should be understood that theaccompanying drawings include all modifications, equivalents andsubstitutions included in the scope and sprit of the present invention.

It will be understood that although the terms first, second, etc., maybe used herein to describe various components, these components shouldnot be limited by these terms. These terms are only used to distinguishone component from another component.

It will be understood that when a component is referred to as being“connected to” or “coupled to” another component, it may be directlyconnected to or coupled to another component or intervening componentsmay be present. In contrast, when a component is referred to as being“directly connected to” or “directly coupled to” another component,there are no intervening components present. In addition, it will beunderstood that when a component is referred to as “controlling” anothercomponent, it may directly control another component, or may alsocontrol another component via the mediation of a third component. Inaddition, it will be understood that when a component is referred to as“providing” another component with information and signals, it maydirectly provide another component with the same and may also provideanother component the same via the mediation of a third component.

As used herein, the singular form is intended to include the pluralforms as well, unless the context clearly indicates otherwise.

In the present application, it will be further understood that the terms“comprises”, includes,” etc. specify the presence of stated features,integers, steps, operations, elements, components, or combinationsthereof, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components, orcombinations thereof.

A vehicle as described in this specification may include all of aninternal combustion engine vehicle including an engine as a powersource, a hybrid vehicle including both an engine and an electric motoras a power source, and an electric vehicle including an electric motoras a power source.

FIG. 1 is a view illustrating the outer appearance of a vehicle 1according to one embodiment of the present invention. For convenience ofdescription, the vehicle 1 is assumed as being a four-wheeled car.

Referring to FIG. 1, the vehicle 1 may include tires 11 a to 11 d whichare rotated by a power source, a steering wheel 12 to adjust thedirection of travel of the vehicle 1, head lamps 13 a and 13 b, wipers14 a and 14 b, and a driver assistance apparatus 100 that will bedescribed below.

The driver assistance apparatus 100 according to the embodiment of thepresent invention may serve to generate a surround-view image of thevehicle 1, to detect information from the generated surround-view image,and to output a control signal to adjust, for example, the direction oftravel of the vehicle 1. At this time, the control signal may beprovided to a controller (770 in FIG. 7), and the controller (770 inFIG. 7) may control, for example, a steering apparatus based on thecontrol signal.

The driver assistance apparatus 100 may include at least one camera, andan image acquired by the camera may be signal-processed in a processor(170 in FIGS. 3A and 3B). For example, as illustrated, a camera 195 maybe mounted to the upper end of a windshield of the vehicle 1 to capturean image of the view in front of the vehicle 1.

Meanwhile, the lowermost point of the body of the vehicle 1 and the roadsurface may be spaced apart from each other by the minimum groundclearance G. This may prevent damage to the vehicle body due to anyobject having a lower height than the minimum ground clearance G.

In addition, the distance between the front left and right tires 11 aand 11 b and the distance between the rear left and right tires 11 c and11 d of the vehicle 1 are assumed as being equal to each other.Hereinafter, the distance between the inner side of the front-wheel lefttire 11 a and the inner side of the front-wheel right tire 11 b and thedistance between the inner side of the rear-wheel left tire 11 c and theinner side of the rear-wheel right tire 11 d are assumed as having thesame value T.

In addition, the full width O of the vehicle 1 is defined as the maximumdistance from the leftmost point to the rightmost point of the body ofthe vehicle 1 excluding side-view mirrors.

Meanwhile, the vehicle 1 illustrated in FIG. 1 may include the driverassistance apparatus 100 that will be described below.

FIGS. 2A to 2C are views referenced to explain cameras attached to thevehicle 1 of FIG. 1 according to the embodiment of the presentinvention.

The driver assistance apparatus 100 including cameras 195 a and 195 b tocapture an image of the view in front of the vehicle 1 will be describedbelow with reference to FIG. 2A.

Although FIG. 2A illustrates the driver assistance apparatus 100 asincluding the two cameras 195 a and 195 b, note that the presentinvention is not limited as to the number of cameras.

Referring to FIG. 2A, the driver assistance apparatus 100 may include afirst camera 195 a having a first lens 193 a and a second camera 195 bhaving a second lens 193 b. In this case, these cameras 195 may bereferred to as stereo cameras.

Meanwhile, the driver assistance apparatus 100 may further include afirst light shield 192 a and a second light shield 192 b, which shieldlight introduced to the first lens 193 a and the second lens 193 brespectively.

The driver assistance apparatus 100 of FIG. 2A may have a structure forattachment or detachment to or from the indoor or outdoor position(e.g., the ceiling or windshield) of the vehicle 1.

The driver assistance apparatus 100 as described above may acquirestereo images of the view in front of the vehicle from the first andsecond cameras 195 a and 195 b. In addition, the driver assistanceapparatus 100 may perform binocular disparity detection based on thestereo images and then perform object detection for at least one stereoimage based on the binocular disparity information. After the objectdetection, the driver assistance apparatus 100 may continuously trackthe movement of an object.

The driver assistance apparatus 100 including cameras 195, 196, 197 and198 to acquire a surround-view image of the vehicle 1 will be describedbelow with reference to FIGS. 2B and 2C.

Although FIGS. 2B and 2C illustrate the driver assistance apparatus 100as including four cameras, note that the present invention is notlimited as to the number of cameras.

Referring to FIGS. 2B and 2C, the driver assistance apparatus 100 mayinclude the cameras 195, 196, 197 and 198. In this case, these cameras195, 196, 197 and 198 may be referred to as so-called around-viewcameras.

The cameras 195, 196, 197 and 198 may be located respectively on thefront side, the left side, the right side, and the rear side of thevehicle 1.

The left camera 196 may be located inside a case enclosing a leftside-view mirror. Alternatively, the left camera 196 may be located atthe exterior of the case enclosing the left side-view mirror. Yetalternatively, the left camera 196 may be located at a region of theexterior of a left front door, a left rear door, or a left fender.

The right camera 197 may be located inside a case enclosing a rightside-view mirror. Alternatively, the right camera 197 may be located atthe exterior of the case enclosing the right side-view mirror. Yetalternatively, the right camera 197 may be located at a region at theexterior of a right front door, a right rear door, or a right fender.

Meanwhile, the rear camera 198 may be located near a rear license plateor a trunk switch.

The front camera 195 may be located near a windshield, near an emblem,or near a radiator grill.

Respective images captured by the cameras 195 to 198 may be transmittedto the processor 170, and the processor 170 may compose the respectiveimages to generate a surround-view image of the vehicle 1.

FIG. 2C illustrates one example of the surround-view image of thevehicle 1. The surround-view image 201 may include a first image region196 i captured by the left camera 196, a second image region 198 icaptured by the rear camera 198, a third image region 197 i captured bythe right camera 197, and a fourth image region 195 i captured by thefront camera 195.

Meanwhile, upon the generation of the surround-view image (hereinafteralso referred to as so-called “around-view image”) from the cameras,boundaries are generated between the respective image regions. Theseboundaries may be subjected to image blending, for natural displaythereof.

Meanwhile, boundary lines 202 a, 202 b, 202 c and 202 d may be displayedat the boundaries between the respective image regions. In addition, thesurround-view image 201 may include a vehicle image at the centerthereof. Here, the vehicle image may be an image generated by theprocessor 170. In addition, the surround-view image 201 of the vehicle 1may be displayed via a display unit 741 of the vehicle 1 or a displayunit 180 of the driver assistance apparatus 100.

FIGS. 3A to 3C are various exemplary internal block diagramsillustrating the driver assistance apparatus 100 according to variousembodiments of the present invention.

The driver assistance apparatus 100 of FIGS. 3A and 3B may generatevehicle associated information via computer vision based signalprocessing of an image received from the camera 195. Here, the vehicleassociated information may include vehicle control information for thedirect control of a vehicle or vehicle traveling assistance informationto guide a vehicle driver during traveling.

Here, the camera 195 may be a monocular camera. Alternatively, thecamera 195 may be the stereo cameras 195 a and 195 b which capture animage of the view in front of the vehicle (hereinafter referred to as a“forward image” of the vehicle). Yet alternatively, the camera 195 maybe included in the around-view cameras 195 to 198 which capture asurround-view image of the vehicle.

FIG. 3A is an internal block diagram of the driver assistance apparatus100 according to the embodiment of the present invention.

Referring to FIG. 3A, the driver assistance apparatus 100 may include aninput unit 110, a communication unit 120, an interface unit 130, amemory 140, a processor 170, a power supply unit 190, a camera 195, adisplay unit 180, and an audio output unit 185.

The input unit 110 is used to receive various inputs from the driver.For example, the input unit 110 may include a plurality of buttons or atouchscreen attached to the cameras 195 to 198. The driver may turn onthe driver assistance apparatus 100 to operate the same using thebuttons or the touchscreen. In addition, the input unit 110 may be usedfor implementation of various other input operations.

The communication unit 120 may exchange data with, for example, a mobileterminal 600, a server 510, or other external appliances of othervehicles in a wireless manner. In particular, the communication unit 120may exchange data with the mobile terminal 600 of the driver in awireless manner. Various wireless data communication protocols such as,for example, Bluetooth, Wi-Fi, Wi-Fi direct, APiX, and NFC may be used.

The communication unit 120 may receive weather information and roadtraffic state information such as, for example, Transport ProtocolExpert Group (TPEG) information, from the mobile terminal 600 or theserver 500. Meanwhile, the communication unit 120 may transmit real-timeinformation, acquired by the driver assistance apparatus 100, to themobile terminal 600 or the server 510.

Meanwhile, when a user gets into the vehicle, the mobile terminal 600 ofthe user may pair with the driver assistance apparatus 100 automaticallyor as the user executes a pairing application.

The communication unit 120 may receive traffic light change informationfrom the external server 510. Here, the external server 510 may be aserver located in a traffic control center.

The interface unit 130 may receive vehicle associated data, orexternally transmit signals processed or generated by the processor 170.To this end, the interface unit 130 may perform data communication with,for example, the controller 770 inside the vehicle, an Audio VideoNavigation (AVN) apparatus 400, and a sensing unit 760 in a wired orwireless communication manner.

The interface unit 130 may receive navigation information via datacommunication with the controller 770, the AVN apparatus 400, or aseparate navigation apparatus. Here, the navigation information mayinclude set destination information, destination based routinginformation, map information related to vehicle traveling, and vehicle'scurrent location information. Meanwhile, the navigation information mayinclude information regarding a vehicle's location on a road.

Meanwhile, the interface unit 130 may receive sensor information fromthe controller 770 or the sensing unit 760.

Here, the sensor information may include at least one selected fromamong vehicle traveling direction information, vehicle locationinformation (GPS information), vehicle angle information, vehicle speedinformation, vehicle acceleration information, vehicle tilt information,vehicle forward/backward movement information, battery information, fuelinformation, tire information, vehicle lamp information, vehicleinterior temperature information, vehicle interior humidity information,and object information.

The sensor information may be acquired from, for example, a headingsensor, a yaw sensor, a gyro sensor, a position module, a vehicleforward/backward movement sensor, a wheel sensor, a vehicle speedsensor, a vehicle body gradient sensor, a battery sensor, a fuel sensor,a tire sensor, a steering sensor based on the rotation of a steeringwheel, a vehicle interior temperature sensor, a vehicle interiorhumidity sensor, and an object sensor (e.g., a radar, Lidar, orultrasonic sensor). Meanwhile, the position module may include a GPSmodule to receive GPS information.

Meanwhile, of the above-specified sensor information, for example,vehicle travel direction information, vehicle location information,vehicle angle information, vehicle speed information, and vehicle tiltinformation, which are related to vehicle traveling, may be referred toas vehicle traveling information.

The interface unit 130 may receive turn-signal information. Here, theturn-signal information may be a turn-on signal of a turn signal lightfor left-turn or right-turn input by the user. When an input to turn ona left or right turn signal light is received via a user input unit (724in FIG. 7) of the vehicle, the interface unit 130 may receiveturn-signal information for left-turn or right-turn.

The interface unit 130 may receive vehicle speed information, steeringwheel rotation angle information, or gearshift information. Theinterface unit 130 may receive vehicle speed information, steering wheelrotation angle information, or gearshift information sensed via thesensing unit 760 of the vehicle. Alternatively, the interface unit 130may receive vehicle speed information, steering wheel rotation angleinformation, or gearshift information from the controller 770 of thevehicle. Meanwhile, here, gearshift information may be informationregarding the current gear position of the vehicle. For example,gearshift information may be information regarding whether the gearshiftis in any one of Park (P), Reverse (R), Neutral (N), and Drive (D), ornumbered gears.

The interface unit 130 may receive user input via the user input unit724 of the vehicle 1. The interface unit 130 may receive user input fromthe input unit 720 of the vehicle 1, or may receive user input by way ofthe controller 770.

The interface unit 130 may receive information acquired from theexternal server 510. The external server 510 may be a server located ina traffic control center. For example, when traffic light changeinformation is received from the external server 510 via a communicationunit 710 of the vehicle, the interface unit 130 may receive the trafficlight change information from the controller (770 of FIG. 7). The memory140 may store various data for the overall operation of the driverassistance apparatus 100 such as, for example, programs for theprocessing or control of the processor 170.

The memory 140 may store data for object verification. For example, whena prescribed object is detected from an image captured by the camera195, the memory 140 may store data to verify, using a prescribedalgorithm, what does the object correspond to.

The memory 140 may store data related to traffic information. Forexample, when prescribed traffic information is detected from an imagecaptured by the camera 195, the memory 140 may store data to verify,using a prescribed algorithm, what does the traffic informationcorrespond to.

Meanwhile, the memory 140 may be any one of various hardware storagedevices such as, for example, a ROM, a RAM, an EPROM, a flash drive, anda hard drive.

The processor 170 controls the overall operation of each unit inside thedriver assistance apparatus 100.

The processor 170 may process a forward image or a surround-view imageof the vehicle acquired by the camera 195. In particular, the processor170 implements computer vision based signal processing. As such, theprocessor 170 may acquire a forward image or a surround-view image ofthe vehicle from the camera 195 and perform object detection and objecttracking based on the image. In particular, the processor 170 mayperform, for example, Lane Detection (LD), Vehicle Detection (VD),Pedestrian Detection (PD), Bright-spot Detection (BD), Traffic SignRecognition (TSR), and road surface detection during object detection.

Meanwhile, a traffic sign may mean prescribed information that may betransmitted to the driver of the vehicle 1. The traffic sign may betransmitted to the driver via a traffic light, a traffic sign, or a roadsurface. For example, the traffic sign may be a go signal or a stopsignal for a vehicle or a pedestrian, which is output from a trafficlight. For example, the traffic sign may be various symbols or textmarked on a traffic sign. For example, the traffic sign may be varioussymbols or text marked on the road surface.

The processor 170 may detect information from a surround-view image ofthe vehicle acquired by the camera 195.

The information may be vehicle traveling situation information. Forexample, the information may include vehicle traveling road information,traffic rule information, adjacent vehicle information, vehicle orpedestrian traffic light information, roadwork information, trafficstate information, parking lot information, and lane information.

The information may be traffic information. The processor 170 may detecttraffic information from any one of a traffic light, a traffic sign, anda road surface included in an image captured by the camera 195. Forexample, the processor 170 may detect a go signal or a stop signal for avehicle or a pedestrian from a traffic light included in an image. Forexample, the processor 170 may detect various symbols or text from atraffic sign included in an image. For example, the processor 170 maydetect various symbols or text from a road surface included in an image.

The processor 170 may verify information by comparing detectedinformation with information stored in the memory 140.

For example, the processor 170 detects a symbol or text indicating aramp from an object included in an acquired image. Here, the object maybe a traffic sign or a road surface. The processor 170 may verify rampinformation by comparing the detected symbol or text with trafficinformation stored in the memory 140.

For example, the processor 170 detects a symbol or text indicatingvehicle or pedestrian stop from an object included in an acquired image.Here, the object may be a traffic sign or a road surface. The processor170 may verify stop information by comparing the detected symbol or textwith traffic information stored in the memory 140. Alternatively, theprocessor 170 detects a stop line from a road surface included in anacquired image. The processor 170 may verify stop information bycomparing the detected stop line with traffic information stored in thememory 140.

For example, the processor 170 may detect whether a traffic lane markeris present from an object included in an acquired image. Here, theobject may be a road surface. The processor 170 may check the color of adetected traffic lane marker. The processor 170 may check whether thedetected traffic lane marker is for a travel lane or a left-turn lane.

For example, the processor 170 may detect vehicle go or stop informationfrom an object included in an acquired image. Here, the object may be avehicle traffic light. Here, the vehicle go information may be a signalto instruct the vehicle to go straight or to turn to the left or right.The vehicle stop information may be a signal to instruct the vehicle tostop. The vehicle go information may be displayed in green and thevehicle stop information may be displayed in red.

For example, the processor 170 may detect pedestrian go or stopinformation from an object included in an acquired image. Here, theobject may be a pedestrian traffic light. Here, the pedestrian goinformation may be a signal to instruct a pedestrian to cross the streetat a crosswalk. The pedestrian stop information may be a signal toinstruct a pedestrian to stop at a crosswalk.

Meanwhile, the processor 170 may control the zoom of the camera 195. Forexample, the processor 170 may control the zoom of the camera 195 basedon an object detection result. When a traffic sign is detected, butcontent written on the traffic sign is not detected, the processor 170may control the camera 195 to zoom in.

Meanwhile, the processor 170 may receive weather information and roadtraffic state information, for example, Transport Protocol Expert Group(TPEG) information via the communication unit 120.

Meanwhile, the processor 170 may recognize, in real time, traffic stateinformation around the vehicle, which has been recognized, based onstereo images, by the driver assistance apparatus 100.

Meanwhile, the processor 170 may receive, for example, navigationinformation from the AVN apparatus 400 or a separate navigationapparatus (not illustrated) via the interface unit 130.

Meanwhile, the processor 170 may receive sensor information from thecontroller 770 or the sensing unit 760 via the interface unit 130. Here,the sensor information may include at least one selected from amongvehicle traveling direction information, vehicle location information(GPS information), vehicle angle information, vehicle speed information,vehicle acceleration information, vehicle tilt information, vehicleforward/backward movement information, battery information, fuelinformation, tire information, vehicle lamp information, vehicleinterior temperature information, vehicle interior humidity information,and steering wheel rotation information.

Meanwhile, the processor 170 may receive navigation information from thecontroller 770, the AVN apparatus 400 or a separate navigation apparatus(not illustrated) via the interface unit 130.

Meanwhile, the processor 170 may be implemented using at least one ofApplication Specific Integrated Circuits (ASICs), Digital SignalProcessors (DSPs), Digital Signal Processing Devices (DSPDs),Programmable Logic Devices (PLDs), Field Programmable Gate Arrays(FPGAs), processors, controllers, micro-controllers, microprocessors,and electric units for implementation of other functions.

The processor 170 may be controlled by the controller 770.

The display unit 180 may display various pieces of information processedin the processor 170. The display unit 180 may display an image relatedto the operation of the driver assistance apparatus 100. To display suchan image, the display unit 180 may include a cluster or a Head UpDisplay (HUD) mounted at the front of the interior of the vehicle.Meanwhile, when the display unit 180 is a HUD, the display unit 180 mayinclude a projector module to project an image to the windshield of thevehicle 1.

The audio output unit 185 may externally output sound based on an audiosignal processed in the processor 170. To this end, the audio outputunit 185 may include at least one speaker.

An audio input unit (not illustrated) may receive user voice. To thisend, the audio input unit may include a microphone. The received voicemay be converted into electrical signals by the audio input unit tothereby be transmitted to the processor 170.

The power supply unit 190 may supply power required to operate therespective components under the control of the processor 170. Inparticular, the power supply unit 190 may receive power from, forexample, a battery inside the vehicle.

The camera 195 acquires a forward image or a surround-view image of thevehicle. The camera 195 may be a monocular camera or the stereo cameras195 a and 195 b to capture a forward image of the vehicle.Alternatively, the camera 195 may be included in the around-view cameras195, 196, 197 and 198 to capture a surround-view image of the vehicle.

The camera 195 may include an image sensor (e.g., a CMOS or a CCD) andan image processing module.

The camera 195 may process a still image or a moving image acquired bythe image sensor. The image processing module may process the stillimage or the moving image acquired by the image sensor. Meanwhile, insome embodiments, the image processing module may be separate from orintegrated with the processor 170.

The camera 195 may acquire an image capturing at least one of a trafficlight, a traffic sign, and a road surface.

The camera 195 may be set to zoom in/out under the control of theprocessor 170. For example, under the control of the processor 170, azoom barrel (not illustrated) included in the camera 195 may be moved tozoom in/out.

The camera 195 may be focused under the control of the processor 170.For example, under the control of the processor 170, a focus barrel (notillustrated) included in the camera 195 may be moved to set a focus. Thefocus may be automatically set based on zoom in/out setting.

Meanwhile, the processor 170 may automatically control the focus tocorrespond to the zoom control of the camera 195.

FIG. 3B is an internal block diagram of the driver assistance apparatus100 according to another embodiment of the present invention.

Referring to FIG. 3B, the driver assistance apparatus 100 has adifference in that it includes the stereo cameras 195 a and 195 b ascompared to the driver assistance apparatus 100 of FIG. 3A. Thefollowing description will focus on this difference.

The driver assistance apparatus 100 may include first and second cameras195 a and 195 b. Here, the first and second cameras 195 a and 195 b maybe referred to as stereo cameras.

The stereo cameras 195 a and 195 b may be configured to be detachablyattached to the ceiling or windshield of the vehicle 1. The stereocameras 195 a and 195 b may respectively include the first lens 193 aand the second lens 193 b.

Meanwhile, the stereo cameras 195 a and 195 b may respectively includethe first light shield 192 a and the second light shield 192 b, whichshield light to be introduced to the first lens 193 a and the secondlens 193 b.

The first camera 195 a captures a first forward image of the vehicle.The second camera 195 b captures a second forward image of the vehicle.The second camera 195 b is spaced apart from the first camera 195 a by aprescribed distance. As the first and second cameras 195 a and 195 b arespaced apart from each other by a prescribed distance, binoculardisparity is generated, which enables the detection of the distance toan object based on binocular disparity.

Meanwhile, when the driver assistance apparatus 100 includes the stereocameras 195 a and 195 b, the processor 170 may implement computer visionbased signal processing. As such, the processor 170 may acquire stereoimages of the view in front of the vehicle from the stereo cameras 195 aand 195 b and perform binocular disparity calculation for the view infront of the vehicle based on the stereo images. Then, the processor 170may perform object detection for at least one of the stereo images basedon the calculated binocular disparity information and, after the objectdetection, continuously track the movement of an object. Here, thestereo images are based on the first forward image received from thefirst camera 195 a and the second forward image received from the secondcamera 195 b.

In particular, the processor 170 may perform, for example, LaneDetection (LD), Vehicle Detection (VD), Pedestrian Detection (PD),Bright-spot Detection (BD), Traffic Sign Recognition (TSR), and roadsurface detection during object detection.

In addition, the processor 170 may perform, for example, calculation ofthe distance to a detected adjacent vehicle, calculation of the speed ofthe detected adjacent vehicle, and calculation of the speed differencewith the detected adjacent vehicle.

The processor 170 may control the zoom of the first and second cameras195 a and 195 b individually. The processor 170 may periodically changethe zoom magnification of the second camera 195 b while fixing the zoomof the first camera 195 a. The processor 170 may periodically change thezoom magnification of the first camera 195 a while fixing the zoom ofthe second camera 195 b.

The processor 170 may control the first or second camera 195 a or 195 bto zoom in or zoom out at a prescribed period.

The processor 170 may set the zoom of the first camera 195 a to a highmagnification so as to be advantageous for object detection at a longdistance. In addition, the processor 170 may set the zoom of the secondcamera 195 b to a low magnification so as to be advantageous for objectdetection at a short distance. At this time, the processor 170 maycontrol the first camera 195 a to zoom in and the second camera 195 b tozoom out.

Conversely, the processor 170 may set the zoom of the first camera 195 ato a low magnification so as to be advantageous for object detection ata short distance. In addition, the processor 170 may set the zoom of thesecond camera 195 b to a high magnification so as to be advantageous forobject detection at a long distance. At this time, the processor 170 maycontrol the first camera 195 a to zoom out and the second camera 195 bto zoom in.

For example, the processor 170 may control the zoom of the first camera195 a or the second camera 195 b according to an object detectionresult. For example, when a traffic sign is detected, but contentwritten on the traffic sign is not detected, the processor 170 maycontrol the first camera 195 a or the second camera 195 b to zoom in.

Meanwhile, the processor 170 may automatically control a focus tocorrespond to the zoom control of the camera 195.

FIG. 3C is an internal block diagram of the driver assistance apparatus100 according to a still another embodiment of the present invention.

Referring to FIG. 3C, the driver assistance apparatus 100 has adifference in that it includes the around-view cameras 195 to 198 ascompared to the driver assistance apparatus 100 of FIG. 3A. Thefollowing description will focus on this difference.

The driver assistance apparatus 100 may include the around-view cameras195 to 198.

Each of the around-view cameras 195 to 198 may include a lens and alight shield configured to shield light to be introduced to the lens.

The around-view cameras may include the left camera 195, the rear camera198, the right camera 197 and the front camera 195.

The front camera 195 captures a forward image of the vehicle. The leftcamera 196 captures a leftward image of the vehicle. The right camera197 captures a rightward image of the vehicle. The rear camera 198captures a rearward image of the vehicle.

The respective images captured by the around-view cameras 195 to 198 aretransmitted to the processor 170.

The processor 170 may generate a surround-view image of the vehicle bycomposing the leftward image, the rearward image, the rightward imageand the forward image of the vehicle. At this time, the surround-viewimage of the vehicle may be a top view image or a bird's eye view image.The processor 170 may receive each of the leftward image, the rearwardimage, the rightward image and the forward image of the vehicle, composethe received images, and convert the composed image into a top viewimage, thereby generating a surround-view image of the vehicle.

Meanwhile, the processor 170 may detect an object based on thesurround-view image of the vehicle. In particular, the processor 170 mayperform, for example, Lane Detection (LD), Vehicle Detection (VD),Pedestrian Detection (PD), Bright-spot Detection (BD), Traffic SignRecognition (TSR), and road surface detection during object detection.

Meanwhile, the processor 170 may control the zoom of the around-viewcameras 195 to 198 individually. The zoom control of the processor 170may be equal to that of the stereo cameras as described above withreference to FIG. 3B.

Some of the components illustrated in FIGS. 3A to 3C may not benecessary in order to implement the driver assistance apparatus 100.Thus, the driver assistance apparatus 100 described in the presentspecification may include a greater or smaller number of components thanthose mentioned above. For example, the driver assistance apparatus 100may include only the processor 170 and the camera 195.

FIGS. 4A and 4B are various exemplary internal block diagramsillustrating the processor of FIGS. 3A and 3B, and FIGS. 5A and 5B areviews referenced to explain the operation of the processor of FIGS. 4Aand 4B.

First, referring to FIG. 4A illustrating one example of the processor170 in internal block diagram, the processor 170 included in the driverassistance apparatus 100 may include an image preprocessor 410, adisparity calculator 420, a segmentation unit 432, an object detector434, an object verification unit 436, an object tracking unit 440, andan application unit 450.

The image preprocessor 410 may receive an image from the camera 195 andpreprocess the received image.

Specifically, the image preprocessor 410 may perform, for example, noisereduction, rectification, calibration, color enhancement, Color SpaceConversion (CSC), interpolation, and camera gain control for the image.As such, the image preprocessor 410 may acquire an image more vivid thanstereo images captured by the camera 195.

The disparity calculator 420 may receive images signal-processed by theimage preprocessor 410, perform stereo matching for the received images,and acquire a binocular disparity map based on the stereo matching. Thatis, the disparity calculator 420 may acquire binocular disparityinformation related to the stereo images for a view in front of thevehicle.

At this time, the stereo matching may be performed on a per pixel basisor on a per prescribed block basis of the stereo images. Meanwhile, thebinocular disparity map may mean a map in which binocular parallaxinformation between stereo images, i.e. left and right images arerepresented by numerical values.

The segmentation unit 432 may perform segmentation and clustering on atleast one of the stereo images based on the binocular disparityinformation from the disparity calculator 420.

Specifically, the segmentation unit 432 may segment at least one of thestereo images into a background and a foreground based on the binoculardisparity information.

For example, the segmentation unit 432 may calculate a region of thedisparity map, in which the binocular disparity information is apredetermined value or less, as a background and exclude thecorresponding region. In this way, a foreground may be relativelyseparated.

In another example, the segmentation unit 432 may calculate a region ofthe disparity map, in which the binocular disparity information is apredetermined value or more, as a foreground and extract thecorresponding region. In this way, the foreground may be separated.

As described above, when the image is segmented into the foreground andthe background based on the binocular disparity information extractedbased on the stereo images, it is possible to reduce a signal processingspeed and a signal processing amount during subsequent object detection.

Subsequently, the object detector 434 may detect an object based onimage segment by the segmentation unit 432.

That is, the object detector 434 may detect an object for at least oneof the stereo images based on the binocular disparity information.

Specifically, the object detector 434 may detect an object for at leastone of the stereo images. For example, the object detector 434 maydetect an object from the foreground separated by image segment.

Subsequently, the object verification unit 436 may classify and verifythe separated object.

To this end, the object verification unit 436 may use, for example, anidentification method using a neural network, a Support Vector Machine(SVM) method, an AdaBoost identification method using a Harr-likefeature, or a Histograms of Oriented Gradients (HOG) method.

Meanwhile, the object verification unit 436 may compare the detectedobject with objects stored in the memory 140 to verify the detectedobject.

For example, the object verification unit 436 may verify an adjacentvehicle, a traffic lane marker, a road surface, a traffic sign, adangerous zone, and a tunnel, which are located around the vehicle.

The object tracking unit 440 may track the verified object. For example,the object tracking unit 440 may verify an object included insequentially acquired stereo images, calculate the motion or motionvector of the verified object, and track, for example, the movement ofthe corresponding object based on the calculated motion or motionvector. As such, the object tracking unit 440 may track, for example, anadjacent vehicle, a traffic lane marker, a road surface, a traffic sign,a dangerous zone, and a tunnel, which are located around the vehicle.

Subsequently, the application unit 450 may calculate, for example, theaccident risk of the vehicle 1 based on various objects located aroundthe vehicle, for example, other vehicles, traffic lane markers, roadsurface, and traffic signs. In addition, the application unit 450 maycalculate the possibility of front-end collision with a front vehicleand whether or not loss of traction occurs.

In addition, the application unit 450 may output, for example, a messageto notify a user of driver assistance information such as, for example,the calculated risk, collision possibility, or traction loss.Alternatively, the application unit 450 may generate a control signal,as vehicle control information, for the attitude control or travelingcontrol of the vehicle 1.

Meanwhile, the image preprocessor 410, the disparity calculator 420, thesegmentation unit 432, the object detector 434, the object verificationunit 436, the object tracking unit 440, and the application unit 450 maybe internal components of an image processing unit 810 included in theprocessor 170 that will be described below with reference to FIG. 7 andthe following drawings.

Meanwhile, in some embodiments, the processor 170 may include only someof the image preprocessor 410, the disparity calculator 420, thesegmentation unit 432, the object detector 434, the object verificationunit 436, the object tracking unit 440, and the application unit 450.For example, when the camera 195 is a monocular camera or around-viewcameras, the disparity calculator 420 may be excluded. In addition, insome embodiments, the segmentation unit 432 may be excluded.

FIG. 4B is an internal block diagram illustrating another example of theprocessor.

Referring to FIG. 4B, the processor 170 includes the same internal unitsas those of the processor 170 of FIG. 4A, but has a signal processingsequence different from that of the processor 170 of FIG. 4A. Thefollowing description will focus on this difference.

The object detector 434 may receive stereo images and detect an objectfor at least one of the stereo images. Differently from FIG. 4A, theobject detector 434 may not detect an object for a segmented image basedon the binocular disparity information, but directly detect an objectfrom the stereo images.

Subsequently, the object verification unit 436 classifies and verifiesthe detected and separated object based on image segment from thesegmentation unit 432 and the object detected by the object detector434.

To this end, the object verification unit 436 may use an identificationmethod using a neural network, a SVM method, an AdaBoost identificationmethod using a Haar-like feature, or a HOG method.

FIGS. 5A and 5B are views referenced to explain an operation method ofthe processor 170 illustrated in FIG. 4A based on stereo images acquiredrespectively from first and second frame periods.

Referring first to FIG. 5A, the stereo cameras 195 a and 195 b acquirestereo images during a first frame period.

The disparity calculator 420 included in the processor 170 receivesstereo images FR1a and FR1b signal-processed by the image preprocessor410 and performs stereo matching for the received stereo images FR1a andFR1b to acquire a disparity map 520.

The disparity map 520 shows a binocular disparity between the stereoimages FR1a and FR1b as levels. As the disparity level is higher, thedistance to the vehicle may be calculated as being shorter. As thedisparity level is lower, the distance to the vehicle may be calculatedas being longer.

Meanwhile, when the disparity map is displayed, the disparity map may bedisplayed with higher brightness as the disparity level is higher anddisplayed with lower brightness as the disparity level is lower.

FIG. 5A shows, by way of example, that, in the disparity map 520, firstto fourth traffic lane markers 528 a, 528 b, 528 c, and 528 d have theirown disparity levels and a roadwork zone 522, a first preceding vehicle524, and a second preceding vehicle 526 have their own disparity levels.

The segmentation unit 432, the object detector 434, and the objectverification unit 436 respectively perform segmentation, objectdetection, and object verification for at least one of the stereo imagesFR1a and FR1b based on the disparity map 520.

FIG. 5A shows, by way of example, that object detection and objectverification for the second stereo image FR1b are performed using thedisparity map 520.

That is, object detection and object verification for first to fourthtraffic lane markers 538 a, 538 b, 538 c, and 538 d, a roadwork zone532, a first preceding vehicle 534, and a second preceding vehicle 536in an image 530 may be performed.

Next, referring to FIG. 5B, the stereo cameras 195 a and 195 b acquirestereo images during a second frame period.

The disparity calculator 420 included in the processor 170 receivesstereo images FR2a and FR2b signal-processed by the image preprocessor410 and performs stereo matching for the received stereo images FR2a andFR2b to acquire a disparity map 540.

FIG. 5B shows, by way of example, that, in the disparity map 540, firstto fourth traffic lane markers 548 a, 548 b, 548 c, and 548 d have theirown disparity levels and a roadwork zone 542, a first preceding vehicle544, and a second preceding vehicle 546 have their own disparity levels.

The segmentation unit 432, the object detector 434, and the objectverification unit 436 respectively perform segmentation, objectdetection, and object verification for at least one of the stereo imagesFR2a and FR2b based on the disparity map 540.

FIG. 5B shows, by way of example, that object detection and objectverification for the second stereo image FR2b are performed using thedisparity map 540.

That is, object detection and object verification for first to fourthtraffic lane markers 558 a, 558 b, 558 c, and 558 d, a roadwork zone552, a first preceding vehicle 554, and a second preceding vehicle 556in an image 550 may be performed.

Meanwhile, the object tracking unit 440 may track verified objects bycomparing FIGS. 5A and 5B with each other.

Specifically, the object tracking unit 440 may track movement of anobject based on the motion or motion vectors of respective objectsverified from FIGS. 5A and 5B. As such, the object tracking unit 440 maytrack, for example, traffic lane markers, a roadwork zone, a firstpreceding vehicle and a second preceding vehicle, which are locatedaround the vehicle.

FIGS. 6A and 6B are views referenced to explain operation of the driverassistance apparatus of FIGS. 3A to 3C.

First, FIG. 6A is a view illustrating the view in front of the vehiclecaptured by the stereo cameras 195 a and 195 b mounted in the vehicle.In particular, FIG. 6A illustrates the view in front of the vehicle in abird's eye view form.

Referring to FIG. 6A, a first traffic lane marker 642 a, a secondtraffic lane marker 644 a, a third traffic lane marker 646 a, and afourth traffic lane marker 648 a are arranged from the left to theright, a roadwork zone 610 a is located between the first traffic lanemarker 642 a and the second traffic lane marker 644 a, a first precedingvehicle 620 a is located between the second traffic lane marker 644 aand the third traffic lane marker 646 a, and a second preceding vehicle630 a is located between the third traffic lane marker 646 a and thefourth traffic lane marker 648 a.

Next, FIG. 6B illustrates the view in front of the vehicle recognized bythe driver assistance apparatus 100 along with various pieces ofinformation. In particular, an image as illustrated in FIG. 6B may bedisplayed on the display unit 180 provided in the driver assistanceapparatus 100, the AVN apparatus 400, or the display unit 741.

Differently from FIG. 6A, FIG. 6B illustrates the display of informationbased on images captured by the stereo cameras 195 a and 195 b.

Referring to FIG. 6B, a first traffic lane marker 642 b, a secondtraffic lane marker 644 b, a third traffic lane marker 646 b, and afourth traffic lane marker 648 b are arranged from the left to theright, a roadwork zone 610 b is located between the first traffic lanemarker 642 b and the second traffic lane marker 644 b, a first precedingvehicle 620 b is located between the second traffic lane marker 644 band the third traffic lane marker 646 b, and a second preceding vehicle630 b is located between the third traffic lane marker 646 b and thefourth traffic lane marker 648 b.

The driver assistance apparatus 100 may verify objects for the roadworkzone 610 b, the first preceding vehicle 620 b, and the second precedingvehicle 630 b by signal processing stereo images captured by the stereocameras 195 a and 195 b. In addition, the driver assistance apparatus100 may verify the first traffic lane marker 642 b, the second trafficlane marker 644 b, the third traffic lane marker 646 b, and the fourthtraffic lane marker 648 b.

Meanwhile, in FIG. 6B, in order to represent object verification for theroadwork zone 610 b, the first preceding vehicle 620 b, and the secondpreceding vehicle 630 b, the roadwork zone 610 b, the first precedingvehicle 620 b, and the second preceding vehicle 630 b are highlighted inblack boxes.

Meanwhile, the driver assistance apparatus 100 may calculate distanceinformation regarding the roadwork zone 610 b, the first precedingvehicle 620 b, and the second preceding vehicle 630 b based on thestereo images captured by the stereo cameras 195 a and 195 b.

FIG. 6B illustrates the display of calculated first distance information611 b, second distance information 621 b, and third distance information631 b, which correspond respectively to the roadwork zone 610 b, thefirst preceding vehicle 620 b, and the second preceding vehicle 630 b.

Meanwhile, the driver assistance apparatus 100 may receive vehiclesensor information from the controller 770 or the sensing unit 760. Inparticular, the driver assistance apparatus 100 may receive vehiclespeed information, gear information, yaw rate information thatrepresents the change rate of a rotation angle (yaw) of the vehicle, andvehicle angle information and display the aforementioned information.

Although FIG. 6B illustrates that vehicle speed information 672, gearinformation 671, and yaw rate information 673 are displayed in an upperportion 670 of a forward image of the vehicle and vehicle angleinformation 682 is displayed in a lower portion 680 of the forward imageof the vehicle, various other examples are possible. In addition,vehicle width information 683 and road curvature information 681 may bedisplayed along with the vehicle angle information 682.

Meanwhile, the driver assistance apparatus 100 may receive speed limitinformation for a road on which the vehicle 700 is traveling via thecommunication unit 120 or the interface unit 130. FIG. 6B illustratesdisplay of speed limit information 640 b.

Although the driver assistance apparatus 100 may display various piecesof information illustrated in FIG. 6B via, for example, the display unit180, alternatively, the driver assistance apparatus 100 may storevarious pieces of information without displaying the same. In addition,the driver assistance apparatus 100 may utilize the information in avariety of applications.

FIG. 7 is a block diagram illustrating one example of the vehicle 1 ofFIG. 1.

The vehicle 1 may include the communication unit 710, the input unit720, the sensing unit 760, an output unit 740, a vehicle drive unit 750,a memory 730, an interface unit 780, the controller 770, the powersource unit 790, the driver assistance apparatus 100, and the AVNapparatus 400.

The communication unit 710 may include one or more modules to enable thewireless communication between the vehicle 1 and the mobile terminal600, between the vehicle 1 and the external server 510, or between thevehicle 1 and another vehicle. In addition, the communication unit 710may include one or more modules to connect the vehicle 1 to one or morenetworks.

The communication unit 710 may include a broadcast receiving module 711,a wireless Internet module 712, a short-range communication module 713,a location information module 714, and an optical communication module715.

The broadcast receiving module 711 is configured to receive a broadcastsignal or broadcast associated information from an external broadcastmanaging server via a broadcast channel. Here, broadcast includes radiobroadcast or TV broadcast.

The wireless Internet module 712 is a module for wireless Internetaccess. The wireless Internet module 712 may be internally or externallycoupled to the vehicle 1. The wireless Internet module 712 may transmitor receive wireless signals via communication networks according towireless Internet technologies.

Examples of such wireless Internet technologies include Wireless LAN(WLAN), Wireless Fidelity (Wi-Fi), Wi-Fi Direct, Digital Living NetworkAlliance (DLNA), Wireless Broadband (WiBro), Worldwide Interoperabilityfor Microwave Access (WiMAX), High Speed Downlink Packet Access (HSDPA),High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), andLTE-A (Long Term Evolution-Advanced). The wireless Internet module 712may transmit and receive data according to one or more of such wirelessInternet technologies, and other Internet technologies as well. Forexample, the wireless Internet module 712 may exchange data with theexternal server 510 in a wireless manner. The wireless Internet module712 may receive weather information and road traffic state information(e.g., Transport Protocol Expert Group (TPEG) information) from theexternal server 510.

The short-range communication module 713 may assist short-rangecommunication using at least one selected from among Bluetooth™, RadioFrequency IDdentification (RFID), Infrared Data Association (IrDA),Ultra-WideBand (UWB), ZigBee, Near Field Communication (NFC),Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, Wireless USB (WirelessUniversal Serial Bus), and the like.

The short-range communication module 713 forms wireless area networks toperform the short-range communication between the vehicle 1 and at leastone external device. For example, the short-range communication module713 may exchange data with the mobile terminal 600 in a wireless manner.The short-range communication module 713 may receive weather informationand road traffic state information (e.g., Transport Protocol ExpertGroup (TPEG) information) from the mobile terminal 600. When the usergets into the vehicle 1, the mobile terminal 600 of the user and thevehicle 1 may pair with each other automatically or as the user executesa pairing application.

The location information module 714 is a module to acquire a location ofthe vehicle 1. A representative example of the location informationmodule 714 includes a Global Position System (GPS) module. For example,when the vehicle utilizes a GPS module, a location of the vehicle may beacquired using signals transmitted from GPS satellites.

The optical communication module 715 may include a light emitting unitand a light receiving unit.

The light receiving unit may convert light into electrical signals toreceive information. The light receiving unit may include Photo Diodes(PDPs) to receive light. The photo diodes may convert light intoelectrical signals. For example, the light receiving unit may receiveinformation regarding a preceding vehicle via light emitted from a lightsource included in the preceding vehicle.

The light emitting unit may include at least one light emitting elementto convert electrical signals into light. Here, the light emittingelement may be a Light Emitting Diode (LED). The light emitting unitconverts electrical signals into light to thereby emit the light. Forexample, the light emitting unit may externally emit light viaflickering of the light emitting element corresponding to a prescribedfrequency. In some embodiments, the light emitting unit may include anarray of a plurality of light emitting elements. In some embodiments,the light emitting unit may be integrated with a lamp provided in thevehicle 1. For example, the light emitting unit may be at least oneselected from among a headlight, a taillight, a brake light, a turnsignal light, and a sidelight. For example, the optical communicationmodule 715 may exchange data with another vehicle 520 via opticalcommunication.

The input unit 720 may include a driving operation unit 721, the camera195, a microphone 723, and the user input unit 724.

The driving operation unit 721 is configured to receive user input forthe driving of the vehicle 1. The driving operation unit 721 may includethe steering input unit 721 a, a shift input unit 721 b, an accelerationinput unit 721 c, and a brake input unit 721 d.

The steering input unit 721 a is configured to receive user input withregard to the direction of travel of the vehicle 1. The steering inputunit 721 a may take the form of the steering wheel 12 as illustrated inFIG. 1. In some embodiments, the steering input unit 721 a may beconfigured as a touchscreen, a touch pad, or a button.

The shift input unit 721 b is configured to receive input for selectingone of Park (P), Drive (D), Neutral (N) and Reverse (R) gears of thevehicle 1 from the user. The shift input unit 721 b may have a leverform. In some embodiments, the shift input unit 721 b may be configuredas a touchscreen, a touch pad, or a button.

The acceleration input unit 721 c is configured to receive user inputfor the acceleration of the vehicle 1. The brake input unit 721 d isconfigured to receive user input for the speed reduction of the vehicle1. Each of the acceleration input unit 721 c and the brake input unit721 d may have a pedal form. In some embodiments, the acceleration inputunit 721 c or the brake input unit 721 d may be configured as atouchscreen, a touch pad, or a button.

The camera 195 may include an image sensor and an image processingmodule. The camera 195 may process a still image or a moving imageacquired by the image sensor (e.g., a CMOS or a CCD). The imageprocessing module may extract required information by processing a stillimage or a moving image acquired via the image sensor and, then, maytransmit the extracted information to the controller 770. Meanwhile, thevehicle 1 may include the camera 195 to capture a forward image or asurround-view image of the vehicle and an internal camera 199 to capturean image of the interior of the vehicle.

The internal camera 199 may capture an image of a passenger. Theinternal camera 199 may capture an image of biometrics of the passenger.

Meanwhile, although FIG. 7 illustrates the camera 195 as being includedin the input unit 720, the camera 195 may be described as being acomponent of the driver assistance apparatus 100 as described above withreference to FIGS. 2 to 6.

The microphone 723 may process external sound signals into electricaldata. The processed data may be utilized in various ways according to afunction that the vehicle 1 is performing. The microphone 723 mayconvert a user voice command into electrical data. The convertedelectrical data may be transmitted to the controller 770.

Meanwhile, in some embodiments, the camera 195 or the microphone 723 maybe components of the sensing unit 760, other than components of theinput unit 720.

The user input unit 724 is configured to receive information from theuser. When information is input via the user input unit 724, thecontroller 770 may control the operation of the vehicle 1 to correspondto the input information. The user input unit 724 may include a touchinput unit or a mechanical input unit. In some embodiments, the userinput unit 724 may be located in a region of the steering wheel. In thiscase, the driver may operate the user input unit 724 with the fingerswhile gripping the steering wheel.

The sensing unit 760 is configured to sense signals associated with, forexample, the traveling of the vehicle 1. To this end, the sensing unit760 may include a collision sensor, a steering sensor, a speed sensor,gradient sensor, a weight sensor, a heading sensor, a yaw sensor, a gyrosensor, a position module, a vehicle forward/backward movement sensor, abattery sensor, a fuel sensor, a tire sensor, a steering sensor based onthe rotation of a steering wheel, a vehicle interior temperature sensor,a vehicle interior humidity sensor, an ultrasonic sensor, an infraredsensor, a radar, and Lidar.

As such, the sensing unit 760 may acquire sensing signals with regardto, for example, vehicle collision information, vehicle travelingdirection information, vehicle location information (GPS information),vehicle angle information, vehicle speed information, vehicleacceleration information, vehicle tilt information, vehicleforward/backward movement information, battery information, fuelinformation, tire information, vehicle lamp information, vehicleinterior temperature information, vehicle interior humidity information,and steering wheel rotation angle information. In addition, the driverassistance apparatus 100 that will be described below may generatecontrol signals for acceleration, speed reduction, direction change andthe like of the vehicle 1 based on surrounding environment informationacquired by at least one of the camera, the ultrasonic sensor, theinfrared sensor, the radar, and Lidar included in the vehicle 1. Here,the surrounding environment information may be information related tovarious objects located within a prescribed distance range from thevehicle 1 that is traveling. For example, the surrounding environmentinformation may include the number of obstacles located within adistance of 100 m from the vehicle 1, the distances to the obstacles,the sizes of the obstacles, the kinds of the obstacles, and the like.

Meanwhile, the sensing unit 760 may further include, for example, anaccelerator pedal sensor, a pressure sensor, an engine speed sensor, anAir Flow-rate Sensor (AFS), an Air Temperature Sensor (ATS), a WaterTemperature Sensor (WTS), a Throttle Position Sensor (TPS), a Top DeadCenter (TDC) sensor, and a Crank Angle Sensor (CAS).

The sensing unit 760 may include a biometric information sensing unit.The biometric information sensing unit is configured to sense andacquire biometric information of the passenger. The biometricinformation may include fingerprint information, iris-scan information,retina-scan information, hand geometry information, facial recognitioninformation, and voice recognition information. The biometricinformation sensing unit may include a sensor to sense biometricinformation of the passenger. Here, the internal camera 199 and themicrophone 723 may operate as sensors. The biometric information sensingunit may acquire hand geometry information and facial recognitioninformation via the internal camera 199.

The output unit 740 is configured to output information processed in thecontroller 770. The output unit 740 may include the display unit 741, asound output unit 742, and a haptic output unit 743.

The display unit 741 may display information processed in the controller770. For example, the display unit 741 may display vehicle associatedinformation. Here, the vehicle associated information may includevehicle control information for the direct control of the vehicle ordriver assistance information to guide vehicle driving. In addition, thevehicle associated information may include vehicle state informationthat notifies a current state of the vehicle or vehicle travelinginformation regarding the traveling of the vehicle.

The display unit 741 may include at least one selected from among aLiquid Crystal Display (LCD), a Thin Film Transistor LCD (TFT LCD), anOrganic Light Emitting Diode (OLED), a flexible display, a 3D display,and an e-ink display.

The display unit 741 may configure an inter-layer structure with a touchsensor, or may be integrally formed with the touch sensor to implement atouchscreen. The touchscreen may function as the user input unit 724which provides an input interface between the vehicle 1 and the user andalso function to provide an output interface between the vehicle 1 andthe user. In this case, the display unit 741 may include a touch sensorwhich senses a touch to the display unit 741 so as to receive a controlcommand in a touch manner. When a touch is input to the display unit 741as described above, the touch sensor may sense the touch and thecontroller 770 may generate a control command corresponding to thetouch. Content input in a touch manner may be characters or numbers, ormay be, for example, instructions in various modes or menu items thatmay be designated.

Meanwhile, the display unit 741 may include a cluster to allow thedriver to check vehicle state information or vehicle travelinginformation while driving the vehicle. The cluster may be located on adashboard. In this case, the driver may check information displayed onthe cluster while looking forward.

Meanwhile, in some embodiments, the display unit 741 may be implementedas a Head Up display (HUD). When the display unit 741 is implemented asa HUD, information may be output via a transparent display provided atthe windshield. Alternatively, the display unit 741 may include aprojector module to output information via an image projected to thewindshield.

The sound output unit 742 is configured to convert electrical signalsfrom the controller 770 into audio signals and to output the audiosignals. To this end, the sound output unit 742 may include, forexample, a speaker. The sound output unit 742 may output soundcorresponding to the operation of the user input unit 724.

The haptic output unit 743 is configured to generate tactile output. Forexample, the haptic output unit 743 may operate to vibrate a steeringwheel, a safety belt, or a seat so as to allow the user to recognize anoutput thereof.

The vehicle drive unit 750 may control the operation of various devicesof the vehicle. The vehicle drive unit 750 may include at least one of apower source drive unit 751, a steering drive unit 752, a brake driveunit 753, a lamp drive unit 754, an air conditioner drive unit 755, awindow drive unit 756, an airbag drive unit 757, a sunroof drive unit758, and a wiper drive unit 759.

The power source drive unit 751 may perform electronic control for apower source inside the vehicle 1. The power source drive unit 751 mayinclude an acceleration device to increase the speed of the vehicle 1and a speed reduction device to reduce the speed of the vehicle 1.

For example, in the case where a fossil fuel based engine (notillustrated) is a power source, the power source drive unit 751 mayperform electronic control for the engine. As such, the power sourcedrive unit 751 may control, for example, an output torque of the engine.In the case where the power source drive unit 751 is the engine, thepower source drive unit 751 may control the speed of the vehicle bycontrolling the output torque of the engine under the control of thecontroller 770.

In another example, when an electric motor (not illustrated) is a powersource, the power source drive unit 751 may perform control for themotor. As such, the power source drive unit 751 may control, forexample, the RPM and torque of the motor.

The steering drive unit 752 may include a steering apparatus. Thus, thesteering drive unit 752 may perform electronic control for a steeringapparatus inside the vehicle 1. For example, the steering drive unit 752may include a steering torque sensor, a steering angle sensor, and asteering motor. The steering torque, applied to the steering wheel 12 bythe driver, may be sensed by the steering torque sensor. The steeringdrive unit 752 may control steering force and a steering angle bychanging the magnitude and direction of current applied to the steeringmotor based on, for example, the speed and the steering torque of thevehicle 1. In addition, the steering drive unit 752 may judge whetherthe direction of travel of the vehicle 1 is correctly being adjustedbased on steering angle information acquired by the steering anglesensor. As such, the steering drive unit 752 may change the direction oftravel of the vehicle 1. In addition, the steering drive unit 752 mayreduce the sense of weight of the steering wheel 12 by increasing thesteering force of the steering motor when the vehicle 1 travels at a lowspeed and may increase the sense of weight of the steering wheel 12 byreducing the steering force of the steering motor when the vehicle 1travels at a high speed. In addition, when the autonomous drivingfunction of the vehicle 1 is executed, the steering drive unit 752 maycontrol the steering motor to generate appropriate steering force basedon, for example, the sensing signals output from the sensing unit 760 orcontrol signals provided by the processor 170 even in the state in whichthe driver operates the steering wheel 12 (i.e. in the state in which nosteering torque is sensed).

The brake drive unit 753 may perform electronic control of a brakeapparatus (not illustrated) inside the vehicle 1. For example, the brakedrive unit 753 may reduce the speed of the vehicle 1 by controlling theoperation of brakes located at wheels. In another example, the brakedrive unit 753 may adjust the direction of travel of the vehicle 1leftward or rightward by differentiating the operation of respectivebrakes located at left and right wheels.

The lamp drive unit 754 may turn at least one lamp arranged inside andoutside the vehicle 1 on or off. The lamp drive unit 754 may include alighting apparatus. In addition, the lamp drive unit 754 may control,for example, the intensity and direction of light of each lamp includedin the lighting apparatus. For example, the lamp drive unit 754 mayperform control for a turn signal lamp, a headlamp or a brake lamp.

The air conditioner drive unit 755 may perform the electronic control ofan air conditioner (not illustrated) inside the vehicle 1. For example,when the interior temperature of the vehicle 1 is high, the airconditioner drive unit 755 may operate the air conditioner to supplycold air to the interior of the vehicle 1.

The window drive unit 756 may perform the electronic control of a windowapparatus inside the vehicle 1. For example, the window drive unit 756may control the opening or closing of left and right windows of thevehicle 1.

The airbag drive unit 757 may perform the electronic control of anairbag apparatus inside the vehicle 1. For example, the airbag driveunit 757 may control an airbag to be deployed in a dangerous situation.

The sunroof drive unit 758 may perform electronic control of a sunroofapparatus inside the vehicle 1. For example, the sunroof drive unit 758may control the opening or closing of a sunroof.

The wiper drive unit 759 may perform the electronic control of thewipers 14 a and 14 b included in the vehicle 1. For example, the wiperdrive unit 759 may perform electronic control with regard to, forexample, the number of operations and the speed of operation of thewipers 14 a and 14 b in response to user input upon receiving the userinput that directs operation of the wipers 14 a and 14 b through theuser input unit 724. In another example, the wiper drive unit 759 mayjudge the amount or strength of rainwater based on sensing signals of arain sensor included in the sensing unit 760 so as to automaticallyoperate the wipers 14 a and 14 b without the user input.

Meanwhile, the vehicle drive unit 750 may further include a suspensiondrive unit (not illustrated). The suspension drive unit may perform theelectronic control of a suspension apparatus (not illustrated) insidethe vehicle 1. For example, when the road surface is uneven, thesuspension drive unit may control the suspension apparatus to reducevibration of the vehicle 1.

The memory 730 is electrically connected to the controller 770. Thememory 730 may store basic data for each unit, control data for theoperation control of the unit, and input/output data. The memory 730 maybe various hardware storage devices such as, for example, a ROM, a RAM,an EPROM, a flash drive, and a hard drive. The memory 730 may storevarious data for the overall operation of the vehicle 1 such as, forexample programs for the processing or control of the controller 770.

The interface unit 780 may serve as a passage for various kinds ofexternal devices that are connected to the vehicle 1. For example, theinterface unit 780 may have a port that is connectable to the mobileterminal 600 and may be connected to the mobile terminal 600 via theport. In this case, the interface unit 780 may exchange data with themobile terminal 600.

Meanwhile, the interface unit 780 may serve as a passage for the supplyof electrical energy to the connected mobile terminal 600. When themobile terminal 600 is electrically connected to the interface unit 780,the interface unit 780 supplies electrical energy from the power sourceunit 790 to the mobile terminal 600 under the control of the controller770.

The controller 770 may control the overall operation of each unit insidethe vehicle 1. The controller 770 may be referred to as an ElectronicControl Unit (ECU).

The controller 770 may be implemented in a hardware manner using atleast one selected from among Application Specific Integrated Circuits(ASICs), Digital Signal Processors (DSPs), Digital Signal ProcessingDevices (DSPDs), Programmable Logic Devices (PLDs), Field ProgrammableGate Arrays (FPGAs), processors, controllers, micro-controllers,microprocessors, and electric units for the implementation of otherfunctions.

The power source unit 790 may supply power required to operate therespective components under the control of the controller 770. Inparticular, the power source unit 790 may receive power from, forexample, a battery (not illustrated) inside the vehicle 1.

The driver assistance apparatus 100 may exchange data with thecontroller 770. A control signal generated in the driver assistanceapparatus 100 may be output to the controller 770. The controller 770may control the direction of travel of the vehicle 1 based on a controlsignal received by the driver assistance apparatus 100.

The AVN apparatus 400 may exchange data with the controller 770. Thecontroller 770 may receive navigation information from the AVN apparatus400 or a separate navigation apparatus (not illustrated). Here, thenavigation information may include set destination information,destination based routing information, and map information or vehiclelocation information related to vehicle traveling.

Meanwhile, some of the components illustrated in FIG. 7 may be notnecessary to implement the vehicle 1. Accordingly, the vehicle 1described in the present specification may include a greater or smallernumber of components than those mentioned above.

Hereinafter, for convenience of description, the driver assistanceapparatus 100 according to the embodiment of the present invention isassumed as being included in the vehicle 1 illustrated in FIG. 1.

FIG. 8 is a flowchart illustrating a control method of the driverassistance apparatus 100 according to one embodiment of the presentinvention.

Referring to FIG. 8, the processor 170 enters an image expansion mode(S800). In the present invention, the image expansion mode means a modethat generates an expanded image using at least a portion of asurround-view image of the vehicle 1 and at least a portion of asurround-view image of another vehicle. Hereinafter, the surround-viewimage of the vehicle 1 is referred to as a “main image” and thesurround-view image of the other vehicle is referred to as a “subimage”. Meanwhile, the main image and the sub image serve to distinguishwhether the subject that generates each image is the vehicle 1 or theother vehicle and are not intended to limit the scope of the presentinvention.

The processor 170 may enter the image expansion mode when apredetermined condition is satisfied.

For example, the processor 170 may enter the image expansion mode whenthe input unit 110 receives user input that commands entry into theimage expansion mode. In this case, the user input may be at least oneof various forms of inputs such as, for example, touch, voice, buttonpush, and a gesture.

In another example, the processor 170 may calculate the degree of riskbased on information regarding the environment around the vehicle 1,acquired by the sensing unit 760 of the vehicle 1, thereby entering theimage expansion mode when the calculated degree of risk exceeds areference value. Upon judging, based on the information regarding theenvironment around the vehicle 1, that various objects such as, forexample, another vehicle approach the vehicle 1 closer than a referencedistance (e.g., 2 m), the processor 170 may enter the image expansionmode in consideration of a high risk of accidents.

In still another example, the processor 170 may enter the imageexpansion mode when the vehicle 1 slows down below a reference speed(e.g., 10 km/h).

In a further example, the processor 170 may enter the image expansionmode when the vehicle 1 begins a parking mode.

Subsequently, the processor 170 generates a main image using one or morecameras 195 to 198 (S805). For example, the processor 170 may turn on aleast one of the cameras 195 to 198 illustrated in FIG. 2 when enteringthe image expansion mode, to generate a main image. That is, the mainimage may include at least one of a forward image, a leftward image, arightward image, and a rearward image of the vehicle 1.

At this time, the main image may have any of various forms. In oneexample, the main image may be a still image or a moving image. Inanother example, the main image may have an around-view form asexemplarily illustrated in FIG. 2C. Hereinafter, for convenience ofdescription, it is assumed that the main image is a so-calledaround-view image including all of the forward image, the leftwardimage, the rightward image and the rearward image of the vehicle 1.

Subsequently, the processor 170 receives a sub image generated by theother vehicle using the communication unit 120 or 710 (S810). That is,the communication unit 120 or 710 receives a sub image from the othervehicle under the control of the processor 170. Here, the sub image isan image generated by cameras provided at the other vehicle.

The communication unit 120 or 170 may directly receive the sub imagefrom the other vehicle. For example, the communication unit 120 or 710may directly receive the sub image from the other vehicle based on avehicle-to-vehicle communication network.

Alternatively, the communication unit 120 or 710 may receive the subimage generated by the other vehicle via the mediation of at least oneexternal device. For example, the other vehicle may transmit the subimage to an external server, and the external server may transmit thesub image received from the other vehicle to the driver assistanceapparatus 100 provided at the vehicle 1.

At this time, the sub image may have any of various forms. In oneexample, the sub image may be a still image or a moving image. Inanother example, the sub image may have an around-view form asexemplarily illustrated in FIG. 2C. Hereinafter, for convenience ofdescription, similar to the main image, it is assumed that the sub imageis a so-called around-view image including all of the forward image, theleftward image, the rightward image and the rearward image of the othervehicle.

In addition, the communication unit 120 or 710 may receive sub imagesgenerated respectively by a plurality of other vehicles. That is, thecommunication unit 120 or 710 may receive a plurality of different subimages.

In this case, the processor 170 may select some of the sub images basedon a predetermined condition or user input. For example, the processor170 may select a sub image showing an obstacle from among the subimages. The selected sub image may be used in Step S815 that will bedescribed below.

In addition, the processor 170 may receive only a sub image generated bya specific vehicle among a plurality of other vehicles using thecommunication unit 120 or 710.

For example, the processor 170 may control the communication unit 120 or710 so as to receive only a sub image generated by another vehicle atthe front of the vehicle 1 from among other vehicles at the front andthe rear of the vehicle 1.

In another example, the processor 170 may control the communication unit120 or 710 so as to receive only a sub image generated by anothervehicle, which is located at a location corresponding to user input or apredetermined condition (e.g., the state of traffic or weather).

Specifically, the communication unit 120 or 710 may transmit an imagerequest signal to another specific vehicle, and receive a sub imagetransmitted from the specific vehicle in response to the image requestsignal under the control of the processor 170.

Meanwhile, the communication unit 120 or 710 may further receivelocation information of the vehicle 1. Thus, the processor 170 may judgewhether a caution zone is present within a predetermined distance fromthe vehicle 1 based on the location information received by thecommunication unit 120 or 710. The caution zone may include, forexample, a zone in which the driver's caution is required such as, forexample, an intersection, an uphill road, a downhill road, a crosswalk,a parking lot, a tunnel, a narrow road, or a curved road. The kinds ofcaution zones may vary according to user input. In this case, theprocessor 170 may control the communication unit 120 or 710 so as totransmit an image request signal to another vehicle located in thecaution zone. Thereby, the communication unit 120 or 710 may receive asub image generated by the other vehicle located in the caution zone,and the processor 170 may generate an expanded image using the subimage. This may advantageously provide the driver of the vehicle 1 withan actual image of the caution zone which is invisible to the driver.

Meanwhile, although FIG. 8 illustrates that Step S805 is followed byStep S810, this is given by way of example. Step S805 and Step S810 maybe performed simultaneously, or Step S810 may be followed by Step S805.

Subsequently, the processor 170 generates an expanded image using themain image and the sub image (S815). That is, the processor 170 maygenerate an expanded image, which has a wider visual field of view thanthe main image, by combining at least a portion of the main image withat least a portion of the sub image.

In this case, the main image and the sub image, included in the expandedimage, may have an overlapping portion. For example, an overlappingportion of the main image and the sub image may be present when there isa common range between the image capture range of the cameras 195 to 198mounted to the vehicle 1 and the image capture range of cameras amountedto the other vehicle.

When the overlapping portion is present in the main image and the subimage, the processor 170 may generate an expanded image by combining themain image and the sub image with each other based on the overlappingportion.

Alternatively, the main image and the sub image, included in theexpanded image, may be spaced apart from each other without anyoverlapping portion. For example, when the other vehicle is locatedoutside the image capture range of the cameras 195 to 198 mounted to thevehicle 1, there is no overlapping portion of the main image and the subimage and, therefore, the resulting expanded image shows the main imageand the sub image spaced apart from each other.

In this case, the processor 170 may generate an expanded image bycombining the main image and the sub image with each other based onlocation information of the vehicle 1 and location information of theother vehicle received by the communication unit 120 or 710. Inaddition, upon combination of the main image and the sub image at whichno overlapping portion is present, the processor 170 may generate anexpanded image by combining the main image and the sub image with eachother based on information regarding the direction of travel of thevehicle 1 and information regarding the direction of travel of the othervehicle. The travel direction information of the vehicle 1 may beacquired by the sensing unit 760 of the vehicle 1, and the traveldirection information of the other vehicle may be received by thecommunication unit 120 or 710.

Meanwhile, the processor 170 may generate an expanded image, either inreal time or periodically. When generating an expanded imageperiodically, the processor 170 may change the period, at which theexpanded image is generated, based on the speed of the vehicle 1. Forexample, when the speed of the vehicle 1 is a first speed, the processor170 may update the expanded image every 3 seconds. When the speed of thevehicle 1 is a second speed which is faster than the first speed, theprocessor 170 may update the expanded image every second.

Although there is a limit, in the related art, in that only an image(i.e. a main image) of an area within the image capture range of thecameras 195 to 198 mounted to the vehicle 1 can be provided to the user,the driver assistance apparatus 100 according to the present inventionmay additionally provide the user with an image (i.e. a sub image) of anarea within the image capture range of cameras mounted to the othervehicle. Consequently, both the safety and convenience of the driveroccupying the vehicle 1 may be simultaneously improved.

Subsequently, the processor 170 executes at least one of predeterminedoperations with relation to the vehicle 1 based on the expanded image(S820).

For example, the processor 170 may generate information regarding anobstacle, which is shown only in the sub image among the main image andthe sub image included in the expanded image. Here, the obstacle shownonly in the sub image may mean an obstacle, which does not fall withinthe visual field of view of the vehicle 1, but falls within the visualfield of view of the other vehicle that provides the sub image. Inaddition, the obstacle information may include various pieces ofinformation related to the obstacle such as, for example, the position,size, color, shape, kind, and movement of the obstacle.

In another example, the processor 170 may generate a control signal thatcommands change of at least one of the speed and direction of thevehicle 1 based on the obstacle information. For example, the controller770 may control the steering drive unit 752, the power source drive unit751, and the brake drive unit 753 based on a control signal provided bythe processor 170. In this way, the risk of accident due to the obstaclemay be reduced compared to the case where only the main image isprovided.

In still another example, the processor 170 may generate a possibletravel route of the vehicle 1 based on the expanded image. Specifically,since the expanded image has a wider visual field of view than the mainimage, the processor 170 may generate a route which connects the currentposition of the vehicle 1 to a position outside of the visual field ofview of the vehicle 1 based on the positional relationship between thevehicle 1 and the other vehicle included in the expanded image.

In this case, there may be a plurality of possible travel routes of thevehicle 1 generated by the processor 170, and the processor 170 mayselect at least one of the routes according to user input or apredetermined order of priority. Alternatively, the processor 170 maygenerate a control signal that commands entry into an autonomous drivingmode with regard to the selected route, and output the control signal tothe controller 770 of the vehicle 1.

In yet another example, the processor 170 may generate a control signalthat commands display of at least one of the expanded image andinformation regarding the expanded image. That is, the display unit 180of the driver assistance apparatus 100, the display unit 741 of thevehicle 1, and/or the AVN apparatus 400 may display only the expandedimage on a screen, may display only the information regarding theexpanded image, or may display both the above, in response to a controlsignal provided from the processor 170.

At this time, the processor 170 may divide a screen of the display unit180 of the driver assistance apparatus 100, the display unit 741 of thevehicle 1, and/or the AVN apparatus 400 into a plurality of sub screens,and may display different pieces of information on the respective subscreens. For example, the processor 170 may display the expanded imageon any one of the sub screens and display the information regarding theexpanded image on another one of the other sub screens.

In addition, the processor 170 may control the display unit 180 of thedriver assistance apparatus 100, the display unit 741 of the vehicle 1,and/or the AVN apparatus 400, in order to provide a certain regionwithin the expanded image, which corresponds to the actual position ofthe obstacle, with prescribed visual effects. For example, the processor170 may control the display unit 180 such that the region within theexpanded image, which corresponds to the actual position of theobstacle, periodically flashes in red.

In addition, the processor 170 may control the display unit 180 of thedriver assistance apparatus 100, the display unit 741 of the vehicle 1,and/or the AVN apparatus 400, in order to rotate and display theexpanded image by an angle corresponding to user input. As such, thedriver may receive help recognizing the environment around the vehicle 1by rotating the expanded image in the clockwise or counterclockwisedirection to match the driver's taste.

In addition, the processor 170 may control the display unit 180 of thedriver assistance apparatus 100, the display unit 741 of the vehicle 1,and/or the AVN apparatus 400, in order to differently display thevehicle 1 and the other vehicle in the expanded image. For example, avehicle image included in the main image may be displayed in red on thedisplay unit 180 and a vehicle image included in the sub image may bedisplayed in blue on the display unit 180.

The driver assistance apparatus 100 is assumed in the followingdescription made with reference to FIGS. 9A to 22B as having entered theimage expansion mode. In addition, the expanded image generated by thedriver assistance apparatus 100 that has entered the image expansionmode and information regarding the expanded image are assumed as beingdisplayed on a screen of the AVN apparatus 400.

FIGS. 9A to 9D are views explaining the operation of the driverassistance apparatus 100 according to one embodiment of the presentinvention.

First, FIG. 9A illustrates the top view of the state in which anothervehicle 2 is in close proximity in front of the vehicle 1. It is assumedthat both the vehicle and the other vehicle 2 are stationary or aretraveling at the same speed in the same lane.

In FIG. 9A, the cameras 195 to 198 mounted to the vehicle 1 capture animage of a first range 911 and cameras mounted to the other vehicle 2capture an image of a second range 921. In this case, as illustrated,there may be an overlapping area 931 where the first range 911 and thesecond range 921 overlap. That is, the overlapping area 931 means anarea that commonly falls within the first range 911 and the second range921 and is photographed by all of the cameras 195 to 198 mounted to thevehicle 1 and the cameras amounted to the other vehicle 2.

FIG. 9B illustrates a main image 941 corresponding to the first range911 illustrated in FIG. 9A. The main image 941, as illustrated, isassumed as being an around-view image. The processor 170 may generatethe main image 941 based on images of the traveling vehicle 1 (e.g., aforward image, a leftward image, a rightward image, and a rearwardimage) generated as the cameras 195 to 198 mounted to the vehicle 1capture an image of the first range 911.

The processor 170 may generate the main image 941 in which an image 942corresponding to the vehicle 1 is centrally located. The image 942corresponding to the vehicle 1, for example, may be directly generatedby the processor 170, or may be previously stored in the memory 140 ofthe driver assistance apparatus 100 or the memory 730 of the vehicle 1.In addition, since the overlapping area 931 is located on the front sideof the first range 911 as illustrated in FIG. 9A, a rear portion of thevehicle body of the other vehicle 2 may be visible in the front portionof the main image 941.

FIG. 9C illustrates a sub image 951 corresponding to the second range921 illustrated in FIG. 9A. The sub image 951, as illustrated, isassumed as being an around-view image like the main image 941. The othervehicle 2 may generate the sub image 951 based on images of thetraveling vehicle 2 (e.g., a forward image, a leftward image, arightward image, and a rearward image) generated by capturing an imageof the second range 921.

In addition, an image 952 corresponding to the other vehicle 2 may belocated at the center of the sub image 951. In addition, since theoverlapping area 931 is located on the rear side of the second range 921as illustrated in FIG. 9A, a front portion of the vehicle body of thevehicle 1 may be visible in the rear portion of the sub image 951.

FIG. 9D is a view illustrating the indoor state of the vehicle 1 in thestate illustrated in FIG. 9A. Referring to FIG. 9D, the driver of thevehicle 1 may keep his/her eyes on the other vehicle 2, which is locatedin front thereof, through the windshield.

The processor 170 may generate an expanded image 961 using the mainimage 941 illustrated in FIG. 9B and the sub image 951 illustrated inFIG. 9C when entering an image expansion mode. For example, theprocessor 170 may generate the expanded image 961 by composing the mainimage 941 and the sub image 951 after changing, for example, thedirection and size of at least one of the main image 941 and the subimage 951 such that an image portion within the main image 941corresponding to the overlapping area 931 illustrated in FIG. 9A and animage portion within the sub image 951 corresponding to the overlappingarea 931 coincide with each other. That is, the driver assistanceapparatus 100 may provide the driver with an image acquired by expandingthe range of the main image 941 forward. In this case, the expandedimage 961 may be an around-view image that seems to capture images ofthe vehicle 1 and the other vehicle 2 from above.

In addition, the processor 170 may generate a control signal for thedisplay of the expanded image 961 and transmit the corresponding controlsignal to at least one of the AVN apparatus 400 equipped in the vehicle1, the display unit 741 of the vehicle 1, and the display unit 180 ofthe driver assistance apparatus 100.

In this way, as exemplarily illustrated in FIG. 9D, the AVN apparatus400 may display the expanded image 961 on a screen based on the controlsignal provided by the processor 170. Although not illustrated, thedisplay unit 741 of the vehicle 1 and the display unit 180 of the driverassistance apparatus 100 may also display the expanded image 961 on ascreen thereof based on the control signal provided by the processor170. At this time, the processor 170 may control the expanded image 961so as to be differently displayed on the display unit 741 of the vehicle1, the display unit 180 of the driver assistance apparatus 100, and theAVN apparatus 400 based on the size and aspect ratio of each screen.

The driver may achieve a visual field of view that is widened forward,compared with the case where only the main image 941 is simply given, byvisually checking the state around the vehicle 1 from the expanded image961 which shows both the main image 941 and the sub image 951.

Meanwhile, the expanded image generated by the processor 170 may providevaluable information regarding blind spot areas of the driver occupyingthe vehicle 1, which will be described further below in more detail.

FIGS. 10A to 10D are views explaining the operation of the driverassistance apparatus 100 according to one embodiment of the presentinvention.

First, FIG. 10A illustrates the top view of the state in which anothervehicle 3 is in close proximity in front of the vehicle 1. It is assumedthat both the vehicle and the other vehicle 3 are stationary or aretraveling at the same speed in the same lane, and that an obstacle 31 ispresent in front of the other vehicle 3.

In FIG. 10A, the cameras 195 to 198 mounted to the vehicle 1 capture animage of a first range 1011 and cameras mounted to the other vehicle 3capture an image of a second range 1021. In this case, as illustrated,there may be an overlapping area 1031 where the first range 1011 and thesecond range 1021 overlap. That is, the overlapping area 1031 may meanan area that commonly falls within the first range 1011 and the secondrange 1021 and is photographed by all of the cameras 195 to 198 mountedto the vehicle 1 and the cameras amounted to the other vehicle 3.

FIG. 10B illustrates a main image 1041 corresponding to the first range1011 illustrated in FIG. 10A. The main image 1041, as illustrated, isassumed as being an around-view image. The processor 170 may generatethe main image 1041 based on images of the traveling vehicle 1 (e.g., aforward image, a leftward image, a rightward image, and a rearwardimage) generated as the cameras 195 to 198 mounted to the vehicle 1capture an image of the first range 1011.

The processor 170 may generate the main image 1041 in which an image1042 corresponding to the vehicle 1 is centrally located. The image 1042corresponding to the vehicle 1, for example, may be directly generatedby the processor 170, or may be previously stored in the memory 140 ofthe driver assistance apparatus 100 or the memory 730 of the vehicle 1.In addition, since the overlapping area 1031 is located on the frontside of the first range 1011 as illustrated in FIG. 10A, a rear portionof the vehicle body of the other vehicle 3 may be visible in the frontportion of the main image 1041.

FIG. 10C illustrates a sub image 1051 corresponding to the second range1021 illustrated in FIG. 10A. The sub image 1051, as illustrated, isassumed as being an around-view image like the main image 1041. Theother vehicle 3 may generate the sub image 1051 based on images of thetraveling vehicle 3 (e.g., a forward image, a leftward image, arightward image, and a rearward image) generated by capturing an imageof the second range 1021.

In addition, an image 1052 corresponding to the other vehicle 3 may belocated at the center of the sub image 1051. In addition, referringagain to FIG. 10A, since the overlapping area 1031 is located on therear side of the second range 1021, a front portion of the vehicle bodyof the vehicle 1 may be visible in the rear portion of the sub image1051.

Meanwhile, differently from the state illustrated in FIG. 9A, in FIG.10A, the obstacle 31 is present in front of the other vehicle 3. Theobstacle 31 falls within the second range 1021 that is the image capturerange of the other vehicle 3. Thus, the obstacle 31 is shown in thefront portion of the sub image 1051 generated by the other vehicle 3.

FIG. 10D is a view illustrating the indoor state of the vehicle 1 in thestate illustrated in FIG. 10A. Referring to FIG. 10D, the driver of thevehicle 1 may keep his/her eyes on the other vehicle 3, which is locatedin front thereof, through the windshield. However, since the obstacle 31is located in front of the other vehicle 3, the driver of the vehicle 1can view only the other vehicle 3 through the windshield and cannot viewthe obstacle 31

The processor 170 may generate an expanded image 1061 using the mainimage 1041 illustrated in FIG. 10B and the sub image 1051 illustrated inFIG. 10C when entering an image expansion mode. For example, theprocessor 170 may generate the expanded image 1061 by composing the mainimage 1041 and the sub image 1051 after changing, for example, thedirection and size of at least one of the main image 1041 and the subimage 1051 such that an image portion within the main image 1041corresponding to the overlapping area 1031 illustrated in FIG. 10A andan image portion within the sub image 1051 corresponding to theoverlapping area 1031 coincide with each other. That is, the driverassistance apparatus 100 may provide the driver with an image acquiredby expanding the range of the main image 1041 forward and rearward. Inthis case, the expanded image 1061 may be an around-view image thatseems to capture images of the vehicle 1 and the other vehicle 3 fromabove.

In addition, the processor 170 may generate a control signal thatcommands display of the expanded image 1061 and may transmit thecorresponding control signal to at least one of the AVN apparatus 400equipped in the vehicle 1, the display unit 741 of the vehicle 1, andthe display unit 180 of the driver assistance apparatus 100.

In this way, as exemplarily illustrated in FIG. 10D, the AVN apparatus400 may display the expanded image 1061 on a screen based on the controlsignal provided by the processor 170. Since the expanded image 1061includes at least a portion of the sub image 1051 and at least a portionof the main image 1041, the driver of the vehicle 1 can check theobstacle 31, which is present at a location that the driver cannot checkwhen viewing only the main image 1041, from the expanded image 1061including the sub image 1051. As a result, the driver of the vehicle 1can, in advance, recognize the obstacle 31, which is hidden by the othervehicle 3 and cannot be checked through the windshield, via the expandedimage 1061. In this way, the driver can carefully perform, for example,the speed reduction, braking, and travel direction change of the vehicle1, which results in a reduction in the risk of accidents.

Although not illustrated, the display unit 741 of the vehicle 1 and thedisplay unit 180 of the driver assistance apparatus 100 may also displaythe expanded image 1061 on the screen thereof based on the controlsignal provided by the processor 170. At this time, the processor 170may control the expanded image 1061 so as to be differently displayed onthe display unit 741 of the vehicle 1, the display unit 180 of thedriver assistance apparatus 100, and the AVN apparatus 400 based on thesize and aspect ratio of each screen.

FIGS. 11A to 11D are views explaining the operation of the driverassistance apparatus 100 according to one embodiment of the presentinvention.

First, FIG. 11A illustrates the top view of the state in which anothervehicle 4 is in close proximity at the lateral side of the vehicle 1.For convenience of description, it is assumed that the other vehicle 4is located at the right side of the vehicle 1, that both the vehicle 1and the other vehicle 4 are stationary or are traveling at the samespeed in two lanes next to each other, and that no object is present infront of the vehicle 1.

In FIG. 11A, the cameras 195 to 198 mounted to the vehicle 1 capture animage of a first range 1111 and cameras mounted to the other vehicle 4capture an image of a second range 1121. In this case, as illustrated,there may be an overlapping area 1131 where the first range 1111 and thesecond range 1121 overlap. That is, the overlapping area 1131 may meanan area that commonly falls within the first range 1111 and the secondrange 1121 and is photographed by all of the cameras 195 to 198 mountedto the vehicle 1 and the cameras amounted to the other vehicle 4.

FIG. 11B illustrates a main image 1141 corresponding to the first range1111 illustrated in FIG. 11A. The main image 1141, as illustrated, isassumed as being an around-view image. The processor 170 may generatethe main image 1141 based on images of the traveling vehicle 1 (e.g., aforward image, a leftward image, a rightward image, and a rearwardimage) generated as the cameras 195 to 198 mounted to the vehicle 1capture an image of the first range 1111.

The processor 170 may generate the main image 1141 in which an image1142 corresponding to the vehicle 1 is centrally located. The image 1142corresponding to the vehicle 1, for example, may be directly generatedby the processor 170, or may be previously stored in the memory 140 ofthe driver assistance apparatus 100 or the memory 730 of the vehicle 1.In addition, since the overlapping area 1131 is located on the rightside of the first range 1111 as illustrated in FIG. 11A, a left portionof the vehicle body of the other vehicle 4 may be visible in the rightportion of the main image 1141.

FIG. 11C illustrates a sub image 1151 corresponding to the second range1121 illustrated in FIG. 11A. The sub image 1151, as illustrated, isassumed as being an around-view image like the main image 1141. Theother vehicle 4 may generate the sub image 1151 based on images of thetraveling vehicle 4 (e.g., a forward image, a leftward image, arightward image, and a rearward image) generated by capturing an imageof the second range 1121.

In addition, an image 1152 corresponding to the other vehicle 4 may belocated at the center of the sub image 1151. In addition, referringagain to FIG. 11A, since the overlapping area 1131 is located on theleft side of the second range 1121, a right portion of the vehicle bodyof the vehicle 1 may be visible in the left portion of the sub image1151.

FIG. 11D is a view illustrating the indoor state of the vehicle 1 in thestate illustrated in FIG. 11A. Referring to FIG. 11D, the driver of thevehicle 1 may keep his/her eyes on the other vehicle 4, which is locatedin front thereof, through the windshield. Referring again to FIG. 11A,the other vehicle 4 is located at the right side of the vehicle 1 and,therefore, the driver can check the other vehicle 4 through thewindshield of the vehicle 1.

The processor 170 may generate an expanded image 1161 using the mainimage 1141 illustrated in FIG. 11B and the sub image 1151 illustrated inFIG. 11C when entering an image expansion mode. For example, theprocessor 170 may generate the expanded image 1161 by composing the mainimage 1141 and the sub image 1151 after changing, for example, thedirection and size of at least one of the main image 1141 and the subimage 1151 such that an image portion within the main image 1141corresponding to the overlapping area 1131 illustrated in FIG. 11A andan image portion within the sub image 1151 corresponding to theoverlapping area 1131 coincide with each other. That is, the driverassistance apparatus 100 may provide the driver with an image acquiredby expanding the range of the main image 1141 leftward and rightward. Inthis case, the expanded image 1161 may be an around-view image thatseems to capture images of the vehicle 1 and the other vehicle 4 fromabove.

In addition, the processor 170 may generate a control signal for thedisplay of the expanded image 1161 and transmit the correspondingcontrol signal to at least one of the AVN apparatus 400 equipped in thevehicle 1, the display unit 741 of the vehicle 1, and the display unit180 of the driver assistance apparatus 100. In this way, as exemplarilyillustrated in FIG. 11D, the AVN apparatus 400 may display the expandedimage 1161 on a screen based on the control signal provided by theprocessor 170. Although not illustrated, the display unit 741 of thevehicle 1 and the display unit 180 of the driver assistance apparatus100 may also display the expanded image 1161 on a screen thereof basedon the control signal provided by the processor 170. At this time, theprocessor 170 may control the expanded image 1161 so as to bedifferently displayed on the display unit 741 of the vehicle 1, thedisplay unit 180 of the driver assistance apparatus 100, and the AVNapparatus 400 based on the size and aspect ratio of each screen.

The driver may achieve a visual field of view that is widened rightward,compared with the case where only the main image 1141 is simply given byvisually checking the state around the vehicle 1 from the expanded image1161 that shows both the main image 1141 and the sub image 1151.

Meanwhile, the expanded image generated by the processor 170 may providevaluable information regarding blind spot areas of the driver occupyingthe vehicle 1, which will be described further below in more detail.

FIGS. 12A to 12D are views explaining the operation of the driverassistance apparatus 100 according to one embodiment of the presentinvention.

First, FIG. 12A illustrates the top view of the state in which anothervehicle 5 is in close proximity at the lateral side of the vehicle 1.For convenience of description, it is assumed that the other vehicle 5is located at the right side of the vehicle 1, that both the vehicle 1and the other vehicle 5 are stationary or are traveling at the samespeed in two lanes next to each other, and that no object is present infront of the vehicle 1.

Meanwhile, differently from the state illustrated in FIG. 11A, apedestrian 41 is present at the right side of the other vehicle 5. Inthis state, the driver of the vehicle 1 has difficulty in checking thepedestrian 41 hidden by the other vehicle 5.

In FIG. 12A, the cameras 195 to 198 mounted to the vehicle 1 capture animage of a first range 1211 and cameras mounted to the other vehicle 5capture an image of a second range 1221. In this case, as illustrated,there may be an overlapping area 1231 where the first range 1211 and thesecond range 1221 overlap. That is, the overlapping area 1231 may meanan area that commonly falls within the first range 1211 and the secondrange 1221 and is photographed by all of the cameras 195 to 198 mountedto the vehicle 1 and the cameras amounted to the other vehicle 5.

FIG. 12B illustrates a main image 1241 corresponding to the first range1211 illustrated in FIG. 12A. The main image 1241, as illustrated, isassumed as being an around-view image. The processor 170 may generatethe main image 1241 based on images of the traveling vehicle 1 (e.g., aforward image, a leftward image, a rightward image, and a rearwardimage) generated as the cameras 195 to 198 mounted to the vehicle 1capture an image of the first range 1211.

The processor 170 may generate the main image 1241 in which an image1242 corresponding to the vehicle 1 is centrally located. The image 1242corresponding to the vehicle 1, for example, may be directly generatedby the processor 170, or may be previously stored in the memory 140 ofthe driver assistance apparatus 100 or the memory 730 of the vehicle 1.In addition, since the overlapping area 1231 is located on the rightside of the first range 1211 as illustrated in FIG. 12A, a left portionof the vehicle body of the other vehicle 5 may be visible in the rightportion of the main image 1241.

FIG. 12C illustrates a sub image 1251 corresponding to the second range1221 illustrated in FIG. 12A. The sub image 1251, as illustrated, isassumed as being an around-view image like the main image 1241. Theother vehicle 5 may generate the sub image 1251 based on images of thetraveling vehicle 5 (e.g., a forward image, a leftward image, arightward image, and a rearward image) generated by capturing an imageof the second range 1221.

In addition, an image 1252 corresponding to the other vehicle 5 may belocated at the center of the sub image 1251. In addition, referringagain to FIG. 12A, since the overlapping area 1231 is located on theleft side of the second range 1221, a right portion of the vehicle bodyof the vehicle 1 may be visible in the left portion of the sub image1251.

Meanwhile, differently from the state illustrated in FIG. 11A, in FIG.12A, the pedestrian 41 is present at the right side of the other vehicle5 and the pedestrian 41 falls within the second range 1221 that is theimage capture range of the other vehicle 5. Thus, the pedestrian 41 isshown in the right portion of the sub image 1251 generated by the othervehicle 5.

FIG. 12D is a view illustrating the indoor state of the vehicle 1 in thestate illustrated in FIG. 12A. Referring to FIG. 12D, the driver of thevehicle 1 has difficulty in checking the pedestrian 41 as well as theother vehicle 5 through the windshield because the pedestrian 41 islocated at the right side of the other vehicle 5.

The processor 170 may generate an expanded image 1261 using the mainimage 1241 illustrated in FIG. 12B and the sub image 1251 illustrated inFIG. 12C when entering an image expansion mode. For example, theprocessor 170 may generate the expanded image 1261 by composing the mainimage 1241 and the sub image 1251 after changing, for example, thedirection and size of at least one of the main image 1241 and the subimage 1251 such that an image portion within the main image 1241corresponding to the overlapping area 1231 illustrated in FIG. 12A andan image portion within the sub image 1251 corresponding to theoverlapping area 1231 coincide with each other. That is, the driverassistance apparatus 100 may provide the driver with an image acquiredby expanding the range of the main image 1241 forward and rearward. Inthis case, the expanded image 1261 may be an around-view image thatseems to capture images of the vehicle 1 and the other vehicle 5 fromabove.

In addition, the processor 170 may generate a control signal for thedisplay of the expanded image 1261 and transmit the correspondingcontrol signal to at least one of the AVN apparatus 400 equipped in thevehicle 1, the display unit 741 of the vehicle 1, and the display unit180 of the driver assistance apparatus 100.

In this way, as exemplarily illustrated in FIG. 12D, the AVN apparatus400 may display the expanded image 1261 on a screen based on the controlsignal provided by the processor 170. Since the expanded image 1261includes at least a portion of the sub image 1251 and at least a portionof the main image 1241, the driver of the vehicle 1 can check thepedestrian 41, who is present at a location that the driver cannot checkwhen viewing only the main image 1241, from the expanded image 1261including the sub image 1251. As a result, the driver of the vehicle 1can, in advance, recognize the pedestrian 41, who is hidden by the othervehicle 5 and cannot be checked via the main image 1241, via theexpanded image 1261. In this way, the driver can carefully perform, forexample, the speed reduction, braking, and travel direction change ofthe vehicle 1, which results in a reduction in the risk of accidents.

Although not illustrated, the display unit 741 of the vehicle 1 and thedisplay unit 180 of the driver assistance apparatus 100 may also displaythe expanded image 1261 on a screen thereof based on the control signalprovided by the processor 170. At this time, the processor 170 maycontrol the expanded image 1261 so as to be differently displayed on thedisplay unit 741 of the vehicle 1, the display unit 180 of the driverassistance apparatus 100, and the AVN apparatus 400 based on the sizeand aspect ratio of each screen.

Meanwhile, although the above description of FIGS. 9A to 12D has focusedon the case where a single vehicle (i.e. another vehicle that generatesa sub image having a common portion with a main image of the vehicle 1)is present close to the vehicle 1, the present invention is not limitedthereto. That is, the driver assistance apparatus 100 according to oneembodiment of the present invention may generate an expanded image inthe case where a plurality of vehicles is present close to the vehicle1. This will be described below in detail.

FIGS. 13A to 13E are views explaining the operation of the driverassistance apparatus 100 according to one embodiment of the presentinvention.

First, FIG. 13A illustrates the top view of a parking lot 1300. Asillustrated, it may be assumed that the vehicle 1, another vehicle 6 atthe left side of the vehicle 1, and the other vehicle 7 at the rightside of the vehicle 1 are located in the parking lot 1300.

In FIG. 13A, the cameras 195 to 198 mounted to the vehicle 1 capture animage of a first range 1311, cameras mounted to the another vehicle 6capture an image of a second range 1321, and cameras mounted to theother vehicle 7 capture an image of a third range 1331.

In this case, as illustrated, there may be a first overlapping area 1341of the first range 1311 and the second range 1321. In addition, theremay be a second overlapping area 1342 of the first range 1311 and thethird range 1331. That is, the first overlapping area 1341 may mean anarea that commonly falls within the first range 1311 and the secondrange 1321 and is photographed by all of the cameras 195 to 198 mountedto the vehicle 1 and the cameras amounted to the another vehicle 6. Inaddition, the second overlapping area 1342 may mean an area thatcommonly falls within the first range 1311 and the third range 1331 andis photographed by all of the cameras 195 to 198 mounted to the vehicle1 and the cameras amounted to the other vehicle 7.

FIG. 13B illustrates a main image 1351 corresponding to the first range1311 illustrated in FIG. 13A. The main image 1351, as illustrated, isassumed as being an around-view image. The processor 170 may generatethe main image 1351 based on images of the traveling vehicle 1 (e.g., aforward image, a leftward image, a rightward image, and a rearwardimage) generated as the cameras 195 to 198 mounted to the vehicle 1capture an image of the first range 1311.

The processor 170 may generate the main image 1351 in which an image1352 corresponding to the vehicle 1 is centrally located. The image 1352corresponding to the vehicle 1, for example, may be directly generatedby the processor 170, or may be previously stored in the memory 140 ofthe driver assistance apparatus 100 or the memory 730 of the vehicle 1.

In addition, as illustrated in FIG. 13A, the first overlapping area 1341is located on the left side of the first range 1311. Thus, asillustrated in FIG. 13B, a right portion of the vehicle body of theanother vehicle 6 may be visible in the left portion of the main image1351. In addition, since the second overlapping area 1342 is located onthe right side of the first range 1311, a left portion of the vehiclebody of the other vehicle 7 may be visible in the right portion of themain image 1351.

FIG. 13C illustrates a first sub image 1361 corresponding to the secondrange 1321 illustrated in FIG. 13A. The first sub image 1361, asillustrated, is assumed as being an around-view image like the mainimage 1351. The another vehicle 6 may generate the first sub image 1361based on images of the traveling vehicle 6 (e.g., a forward image, aleftward image, a rightward image, and a rearward image) generated bycapturing an image of the second range 1321.

In addition, an image 1362 corresponding to the another vehicle 6 may belocated at the center of the sub image 1361. In addition, as illustratedin FIG. 13A, since the first overlapping area 1341 is located on theright side of the second range 1321, a left portion of the vehicle bodyof the vehicle 1 may be visible in the right portion of the first subimage 1361.

FIG. 13D illustrates a second sub image 1371 corresponding to the thirdrange 1331 illustrated in FIG. 13A. The second sub image 1371, asillustrated, is assumed as being an around-view image like the mainimage 1351. The other vehicle 7 may generate the second sub image 1371based on images of the traveling vehicle 7 (e.g., a forward image, aleftward image, a rightward image, and a rearward image) generated bycapturing an image of the third range 1331.

In addition, an image 1372 corresponding to the other vehicle 7 may belocated at the center of the second sub image 1371. In addition, asillustrated in FIG. 13A, since the second overlapping area 1342 islocated on the left side of the third range 1331, a right portion of thevehicle body of the vehicle 1 may be visible in the left portion of thesecond sub image 1371.

FIG. 13E is a view illustrating the indoor state of the vehicle 1 in thestate illustrated in FIG. 13A. The processor 170 may generate anexpanded image 1381 using the main image 1351 illustrated in FIG. 13B,the first sub image 1361 illustrated in FIG. 13C, and the second subimage 1371 illustrated in FIG. 13D when entering an image expansionmode.

For example, the processor 170 may compose the main image 1351 and thefirst sub image 1361 with each other after changing, for example, thedirection and size of at least one of the main image 1351 and the firstsub image 1361 such that an image portion within the main image 1351corresponding to the first overlapping area 1341 illustrated in FIG. 13Aand an image portion within the first sub image 1361 corresponding tothe first overlapping area 1341 coincide with each other. In addition,the processor 170 may combine the main image 1351 and the second subimage 1371 with each other after changing, for example, the directionand size of at least one of the main image 1351 and the second sub image1371 such that an image portion within the main image 1351 correspondingto the second overlapping area 1342 illustrated in FIG. 13A and an imageportion within the second sub image 1371 corresponding to the secondoverlapping area 1342 coincide with each other.

In one example, assuming that a parking line delimiting the parking lot1300 has a constant width and the width of a parking line shown in themain image 1351 is two times the width of a parking line shown in thefirst sub image 1361, the processor 170 may first reduce the size of themain image 1351 to a half, or may double the size of the first sub image1361 and, thereafter, may combine the main image 1351 and the first subimage 1361 with each other. In addition, the processor 170 may alsocombine the second sub image 1371 with the main image 1351 with eachother in the same manner. In this way, the processor 170 may generatethe expanded image 1381 by combining images generated by differentvehicles without any sense of discontinuity.

Consequently, the driver assistance apparatus 100 may provide the driverwith an expanded image acquired by expanding the range of the main image1351 leftward and rightward. In this case, the expanded image 1381 maybe an around-view image that seems to capture images of the vehicle 1and the other two vehicles 6 and 7 from above.

In addition, the processor 170 may generate a control signal for thedisplay of the expanded image 1381 and transmit the correspondingcontrol signal to at least one of the AVN apparatus 400 equipped in thevehicle 1, the display unit 741 of the vehicle 1, and the display unit180 of the driver assistance apparatus 100. In this way, as exemplarilyillustrated in FIG. 13E, the AVN apparatus 400 may display the expandedimage 1381 on a screen based on the control signal provided by theprocessor 170. Although not illustrated, the display unit 741 of thevehicle 1 and the display unit 180 of the driver assistance apparatus100 may also display the expanded image 1381 on a screen thereof basedon the control signal provided by the processor 170. At this time, theprocessor 170 may control the expanded image 1381 so as to bedifferently displayed on the display unit 741 of the vehicle 1, thedisplay unit 180 of the driver assistance apparatus 100, and the AVNapparatus 400 based on the size and aspect ratio of each screen.

The driver may achieve a visual field of view that is widened leftwardand rightward, compared with the case where only the main image 1351 issimply given by visually checking the state around the vehicle 1 fromthe expanded image 1381 that shows the main image 1351, the first subimage 1361, and the second sub image 1371.

Meanwhile, although the above description of FIGS. 13A to 13E hasfocused on the case where the two vehicles 6 and 7 are located at theleft and right sides of the vehicle 1, this is given by way of exampleand is not intended to limit the scope of the present invention. Forexample, an expanded image may be generated using the above-describedmethod even when the two vehicles 6 and 7 are located at the front andrear sides of the vehicle 1. In another example, an expanded image maybe generated using the above-described method even when one vehicle 6 islocated at the left side of the vehicle 1 and the other vehicle 7 islocated at the rear side of the vehicle 1.

Meanwhile, the expanded image generated by the processor 170 may providevaluable information regarding blind spot areas of the driver occupyingthe vehicle 1, which will be described further below in more detail.

FIGS. 14A to 14E are views explaining the operation of the driverassistance apparatus 100 according to one embodiment of the presentinvention.

First, FIG. 14A illustrates the top view of a parking lot 1400. Asillustrated, it may be assumed that the vehicle 1, another vehicle 8 atthe left side of the vehicle 1, and the other vehicle 9 at the rightside of the vehicle 1 are located in the parking lot 1400. ComparingFIG. 14A with FIG. 13A, there is a difference in that a pedestrian 51who is near the other vehicle 9 is moving toward the vehicle 1.

In FIG. 14A, the cameras 195 to 198 mounted to the vehicle 1 capture animage of a first range 1411, cameras mounted to the another vehicle 8capture an image of a second range 1421, and cameras mounted to theother vehicle 9 capture an image of a third range 1431.

In this case, as illustrated, there may be a first overlapping area 1441of the first range 1411 and the second range 1421. In addition, theremay be a second overlapping area 1442 of the first range 1411 and thethird range 1431. That is, the first overlapping area 1441 may mean anarea that commonly falls within the first range 1411 and the secondrange 1421 and is photographed by all of the cameras 195 to 198 mountedto the vehicle 1 and the cameras amounted to the another vehicle 8. Inaddition, the second overlapping area 1442 may mean an area thatcommonly falls within the first range 1411 and the third range 1431 andis photographed by all of the cameras 195 to 198 mounted to the vehicle1 and the cameras amounted to the other vehicle 9.

FIG. 14B illustrates a main image 1451 corresponding to the first range1411 illustrated in FIG. 14A. The main image 1451, as illustrated, isassumed as being an around-view image. The processor 170 may generatethe main image 1451 based on images of the traveling vehicle 1 (e.g., aforward image, a leftward image, a rightward image, and a rearwardimage) generated as the cameras 195 to 198 mounted to the vehicle 1capture an image of the first range 1411.

The processor 170 may generate the main image 1451 in which an image1452 corresponding to the vehicle 1 is centrally located. The image 1452corresponding to the vehicle 1, for example, may be directly generatedby the processor 170, or may be previously stored in the memory 140 ofthe driver assistance apparatus 100 or the memory 730 of the vehicle 1.

In addition, as illustrated in FIG. 14A, the first overlapping area 1441is located on the left side of the first range 1411. Thus, asillustrated in FIG. 14B, a right portion of the vehicle body of theanother vehicle 8 may be visible in the left portion of the main image1451. In addition, since the second overlapping area 1442 is located onthe right side of the first range 1411, a left portion of the vehiclebody of the other vehicle 9 may be visible in the right portion of themain image 1451.

However, the pedestrian 51 is not shown in the main image 1451 and ishidden by the other vehicle 9. Thus, when the driver of the vehicle 1attempts to get out the parking lot 1400 by depending on only the mainimage 1451, there is the risk of collision with the pedestrian 51 who ismoving toward the vehicle 1.

FIG. 14C illustrates a first sub image 1461 corresponding to the secondrange 1421 illustrated in FIG. 14A. The first sub image 1461, asillustrated, is assumed as being an around-view image like the mainimage 1451. The another vehicle 8 may generate the first sub image 1461based on images of the traveling vehicle 8 (e.g., a forward image, aleftward image, a rightward image, and a rearward image) generated bycapturing an image of the second range 1421.

In addition, an image 1462 corresponding to the another vehicle 8 may belocated at the center of the first sub image 1461. In addition, asillustrated in FIG. 14A, since the first overlapping area 1441 islocated on the right side of the second range 1421, a left portion ofthe vehicle body of the vehicle 1 may be visible in the right portion ofthe first sub image 1461.

FIG. 14D illustrates a second sub image 1471 corresponding to the thirdrange 1431 illustrated in FIG. 14A. The second sub image 1471, asillustrated, is assumed as being an around-view image like the mainimage 1451. The other vehicle 9 may generate the second sub image 1471based on images of the traveling vehicle 9 (e.g., a forward image, aleftward image, a rightward image, and a rearward image) generated bycapturing an image of the third range 1431.

In addition, an image 1472 corresponding to the other vehicle 9 may belocated at the center of the second sub image 1471. In addition, asillustrated in FIG. 14A, since the second overlapping area 1442 islocated on the left side of the third range 1431, a right portion of thevehicle body of the vehicle 1 may be visible in the left portion of thesecond sub image 1471.

FIG. 14E is a view illustrating the indoor state of the vehicle 1 in thestate illustrated in FIG. 14A. The processor 170 may generate anexpanded image 1381 using the main image 1451 illustrated in FIG. 14B,the first sub image 1461 illustrated in FIG. 14C, and the second subimage 1471 illustrated in FIG. 14D when entering an image expansionmode.

For example, the processor 170 may compose the main image 1451 and thefirst sub image 1461 with each other after changing, for example, thedirection and size of at least one of the main image 1451 and the firstsub image 1461 such that an image portion within the main image 1451corresponding to the first overlapping area 1441 illustrated in FIG. 14Aand an image portion within the first sub image 1461 corresponding tothe first overlapping area 1441 coincide with each other. In addition,the processor 170 may compose the main image 1451 and the second subimage 1471 with each other after changing, for example, the directionand size of at least one of the main image 1451 and the second sub image1471 such that an image portion within the main image 1451 correspondingto the second overlapping area 1442 illustrated in FIG. 14A and an imageportion within the second sub image 1471 corresponding to the secondoverlapping area 1442 coincide with each other.

In one example, assuming that a parking line delimiting the parking lot1400 has a constant width and the width of a parking line shown in themain image 1451 is two times the width of a parking line shown in thefirst sub image 1461, the processor 170 may first reduce the size of themain image 1451 to a half, or may double the size of the first sub image1461 and, thereafter, may combine the main image 1451 and the first subimage 1461 with each other. In addition, the processor 170 may alsocombine the second sub image 1471 with the main image 1451 in the samemanner. In this way, the processor 170 may generate the expanded image1481 by combining images generated by different vehicles without anysense of discontinuity.

Consequently, the driver assistance apparatus 100 may provide the driverwith an expanded image acquired by expanding the range of the main image1451 leftward and rightward. In this case, the expanded image 1481 maybe an around-view image that seems to capture images of the vehicle 1and the other two vehicles 8 and 9 from above.

In addition, the processor 170 may generate a control signal for thedisplay of the expanded image 1481 and transmit the correspondingcontrol signal to at least one of the AVN apparatus 400 equipped in thevehicle 1, the display unit 741 of the vehicle 1, and the display unit180 of the driver assistance apparatus 100. In this way, as exemplarilyillustrated in FIG. 14E, the AVN apparatus 400 may display the expandedimage 1481 on a screen based on the control signal provided by theprocessor 170.

Although not illustrated, the display unit 741 of the vehicle 1 and thedisplay unit 180 of the driver assistance apparatus 100 may also displaythe expanded image 1481 on a screen thereof based on the control signalprovided by the processor 170. At this time, the processor 170 maycontrol the expanded image 1481 so as to be differently displayed on thedisplay unit 741 of the vehicle 1, the display unit 180 of the driverassistance apparatus 100, and the AVN apparatus 400 based on the sizeand aspect ratio of each screen.

The driver may achieve a visual field of view that is widened leftwardand rightward, compared with the case where only the main image 1451 issimply given by visually checking the state around the vehicle 1 fromthe expanded image 1481 that shows the main image 1451, the first subimage 1461, and the second sub image 1471. In particular, the driver ofthe vehicle 1 can recognize, via the expanded image 1481 displayed on,for example, the AVN apparatus 400, an object such as, for example, thepedestrian 51, who is present at a location outside the visual field ofview of the driver occupying the vehicle 1 or at a location that isinvisible from the main image 1451.

Although the above description of FIGS. 13A to 14E has focused on thecase where two vehicles are located close to the vehicle 1, it will beclearly understood by those skilled in the art that the driverassistance apparatus 100 may generate an expanded image in theabove-described manner even when three or more vehicles are presentclose to the vehicle 1.

Meanwhile, the driver assistance apparatus 100 according to oneembodiment of the present invention may generate an expanded image usinga sub image generated by another vehicle that is not close to thevehicle 1. This will be described below in detail.

FIGS. 15A to 15C are views explaining the operation of the driverassistance apparatus 100 according to one embodiment of the presentinvention.

First, FIG. 15A illustrates the top view of an intersection 1500. Asillustrated, it may be assumed that the vehicle 1 and three differentvehicles 10, 11 and 12 are traveling in the vicinity of the intersection1500.

In FIG. 15A, the cameras 195 to 198 mounted to the vehicle 1 capture animage of a first range 1511, cameras mounted to another vehicle 10capture an image of a second range 1521, cameras mounted to stillanother vehicle 11 capture an image of a third range 1531, and camerasmounted to yet another vehicle 12 capture an image of a fourth range1541.

In this case, as illustrated, the first range 1511 may have nooverlapping area with the second to fourth ranges 1521, 1531 and 1541.On the other hand, the another vehicle 10 and the still another vehicle11 are close to each other and there may be an overlapping area 1571where the second range 1521 and the third range 1531 overlap.

FIG. 15B illustrates a screen of the AVN apparatus 400 provided at thevehicle 1 in the state illustrated in FIG. 15A. It is assumed that thescreen of the AVN apparatus 400 is provided with a touch sensor toreceive touch input of the driver.

A navigation image 1510 may be displayed in a section, corresponding tothe current location of the vehicle 1, on the screen of the AVNapparatus 400. For example, the processor 170 may match locationinformation of the vehicle 1, received by the communication unit 120 or710 of the vehicle 1, with an electronic map stored in the memory 140 ofthe driver assistance apparatus 100 or the memory 730 of the vehicle 1,thereby providing the AVN apparatus 400 with a control signal to displaythe navigation image 1510 of the matched map portion. At this time, thenavigation image 1510 may display an indicator 1511 which guides thecurrent location of the vehicle 1.

Meanwhile, there is no vehicle close to the vehicle 1 in theintersection 1500 and, therefore, the driver assistance apparatus 100cannot receive a sub image, which has a common portion with the mainimage 1551 corresponding to the first range 1511.

In this case, the driver assistance apparatus 100 may select at leastone of other vehicles distant from the vehicle 1, i.e. the othervehicles 10, 11 and 12 which capture images of the second to fourthranges 1521, 1531 and 1541 having no overlapping area with the firstrange 1511, and then receive a sub image generated by the selectedvehicle.

The driver assistance apparatus 100 may select another vehicle which islocated at a location distant from the vehicle 1 based on user input.

Referring again to FIG. 15B, the driver may touch a point P1 within theoverall region of the navigation image 1510 displayed on the screen ofthe AVN apparatus 400. At this time, prescribed visual effects may occuron the navigation image 1510 as the point P1 is touched. The processor170 may check the two other vehicles 10 and 11, which are traveling inan area of the intersection 1500 corresponding to a region including thetouched point P1 within the overall region of the navigation image 1510.

For example, the processor 170 may acquire GPS coordinate values withinthe electronic map, which match with the touched point P1. Subsequently,the processor 170 may transmit an image request signal to the two othervehicles 10 and 11 traveling at the actual location of the intersection1500 which corresponds to the GPS coordinate values, through the use ofthe communication unit 120 or 710. In addition, the processor 170 mayreceive sub images transmitted by the two other vehicles 10 and 11 inresponse to the image request signal, through the use of thecommunication unit 120 or 710.

FIG. 15C illustrates the screen of the AVN apparatus 400 displaying anexpanded image 1520. The processor 170 may divide the screen of the AVNapparatus 400 into two or more sub screens, and display different piecesof information on a per sub screen basis. As illustrated, the processor170 may divide the screen of the AVN apparatus 400 into a first subscreen S1 and a second sub screen S2 and generate a control signal todisplay the navigation image 1510 on the first sub screen S1 and theexpanded image 1520 on the second sub screen S2. Although FIG. 15Cillustrates only the AVN apparatus 400, it will be clearly understood bythose skilled in the art that the display unit 741 of the vehicle 1 andthe display unit 180 of the driver assistance apparatus 100 may alsoundergo screen division into a plurality of sub screens based on acontrol signal provided by the processor 170.

Meanwhile, since the image capture ranges 1521 and 1531 of the two othervehicles 10 and 11 have the overlapping area 1571, a sub image 1561included in the expanded image 1520 may show an image 1562 correspondingto the another vehicle and an image 1572 corresponding to the stillanother vehicle 11.

In addition, the second range 1521 and the third range 1531, which arethe image capture ranges of the two other vehicles 10 and 11, have nooverlapping area with the first range 1511. Thus, the main image 1551and the sub image 1561 may be shown in the expanded image 1520 as beingseparate from each other.

Specifically, the processor 170 may judge the direction of the two othervehicles 10 and 11 relative to the vehicle 1 and the distance from thetwo other vehicles 10 and 11 to the vehicle 1 based on locationinformation of the vehicle 1 and location information of the two othervehicles 10 and 11. In addition, the processor 170 may generate theexpanded image 1520 in which the main image 1551 and the sub image 1561are spaced apart from each other by a prescribed distance in aprescribed direction based on the judged results.

FIGS. 16A to 16D are views explaining the operation of the driverassistance apparatus 100 according to one embodiment of the presentinvention.

The driver assistance apparatus 100 according to the present inventionmay judge whether a caution zone is present within a predetermineddistance (e.g., 300 m) from the vehicle 1, and generate an expandedimage using a sub image generated by at least one vehicle located in thecaution zone.

Specifically, when the communication unit 120 or 710 receives locationinformation of the vehicle 1 (e.g., GPS coordinate values), theprocessor 170 may judge whether a caution zone is present within apredetermined distance from the vehicle 1 based on correspondinglocation information and an electronic map stored in the memory 140 or730. For example, the electronic map may include various pieces offormation related to routes, along which the vehicle 1 can travel, suchas, for example, road information (e.g., speed bumps, road width, andtraffic signs) and information regarding facilities (e.g., hospitals,country clubs, and parks). As such, the processor 170 may recognize acaution zone within a predetermined distance from the GPS coordinatevalues of the vehicle 1 among various pieces of information included inthe electronic map.

Here, the caution zone may mean a zone where the driver of the vehicle 1has to pay more attention than during ordinary driving. Examples ofcaution zones may include an intersection, an uphill road, a downhillroad, a crosswalk, a parking lot, a tunnel, a narrow road, a curvedroad, and a pothole. The processor 170 may change the number, the kind,and the order of priority of caution zones according to user input.

When a plurality of caution zones is present within a prescribeddistance from the vehicle 1, the processor 170 may make known only aprescribed number of high-rank caution zones in the order of priority.

In addition, when a plurality of caution zones is present within aprescribed distance from the vehicle 1, under the control of theprocessor 170, the caution zones may be visually differently displayedon, for example, a navigation system screen based on the order ofpriority. For example, the highest priority caution zone may bedisplayed in red, and the lowest priority caution zone may be displayedin blue.

In addition, the processor 170 may automatically enter an imageexpansion mode when at least one caution zone is present within aprescribed distance from the vehicle 1.

FIG. 16A illustrates one crosswalk 1601 as one exemplary caution zone.The crosswalk 1601 is a road area where a pedestrian crosses. In FIG.16A, the vehicle 1 may generate a main image corresponding to a firstrange 1611 and another vehicle 13 may generate a sub image correspondingto a second range 1621.

Since the crosswalk 1601 is not included in the first range 1611, thedriver cannot check the real-time state of the crosswalk 1601 only usingthe main image corresponding to the first range 1611. On the other hand,since the second range 1621 includes at least a portion of the crosswalk1601 and a pedestrian 61 near the crosswalk 1601, the pedestrian 61 whowaits to cross the crosswalk 1601 is shown in the sub imagecorresponding to the second range 1621.

The processor 170 may generate an expanded image including the mainimage corresponding to the first range 1611 and the sub imagecorresponding to the second range 1621, thereby assisting the driver ofthe vehicle 1 in checking the pedestrian 61 in advance even at alocation distant from the crosswalk 1601.

FIG. 16B illustrates a curved road 1602 as another exemplary cautionzone. The curved road 1602 may be a road area, the curvature of which isa prescribed value or more. The curved road 1602 may cause blind spotsthat are invisible to the driver of the vehicle 1 according to themagnitude of curvature. In FIG. 16B, the vehicle 1 may generate a mainimage corresponding to a first range 1612 and another vehicle 13 maygenerate a sub image corresponding to a second range 1622.

Since the curved road 1602 is not included in the first range 1612, thedriver cannot check the real-time state of the curved road 1602 onlyusing the main image corresponding to the first range 1612. On the otherhand, since the second range 1622 includes at least a portion of thecurved road 1602 and an obstacle 62 on the curved road 1602, theobstacle 62 which has the possibility of collision with the vehicle 1 isshown in the sub image corresponding to the second range 1622.

The processor 170 may generate an expanded image including the mainimage corresponding to the first range 1612 and the sub imagecorresponding to the second range 1622, thereby assisting the driver ofthe vehicle 1 in checking the obstacle 62 in advance even at a locationdistant from the curved road 1602.

FIG. 16C illustrates a downhill road 1603 as one exemplary caution zone.The downhill road 1603 is a steep road extending from a high altitude toa low altitude. In FIG. 16C, the vehicle 1 may generate a main imagecorresponding to a first range 1613 and another vehicle 13 may generatea sub image corresponding to a second range 1623.

Since the downhill road 1603 is not included in the first range 1613,the driver cannot check the real-time state of the downhill road 1603only using the main image corresponding to the first range 1613. On theother hand, since the second range 1623 includes at least a portion ofthe downhill road 1603 and an obstacle 63 on the downhill road 1603, theobstacle 63 which has the possibility of collision with the vehicle 1 isshown in the sub image corresponding to the second range 1623.

The processor 170 may generate an expanded image including the mainimage corresponding to the first range 1613 and the sub imagecorresponding to the second range 1623, thereby assisting the driver ofthe vehicle 1 in checking the obstacle 63 in advance even at a locationdistant from the downhill road 1603.

FIG. 16D illustrates a tunnel 1604 as one exemplary caution zone. InFIG. 16D, the vehicle 1 may generate a main image corresponding to afirst range 1614 and another vehicle 14 may generate a sub imagecorresponding to a second range 1624.

Since the tunnel 1604 is not included in the first range 1614, thedriver cannot check the real-time state of the tunnel 1604 via only themain image corresponding to the first range 1614. On the other hand,since the second range 1624 includes at least a portion of the tunnel1604 and a pothole 64 present in the tunnel 1604, the pothole 64 whichmay impair the vehicle 1 is shown in the sub image corresponding to thesecond range 1624.

The processor 170 may generate an expanded image including the mainimage corresponding to the first range 1614 and the sub imagecorresponding to the second range 1624, thereby assisting the driver ofthe vehicle 1 in checking the pothole 64 in advance even at a locationdistant from the tunnel 1604.

FIGS. 17A and 17B are views explaining the operation of the driverassistance apparatus 100 according to one embodiment of the presentinvention.

The driver assistance apparatus 100 according to the present inventionmay judge whether a congested zone 1700 is present within apredetermined distance (e.g., 300 m) from the vehicle 1 or within anarea selected by the driver, and generate an expanded image using a subimage generated by at least one vehicle located in the congested zone1700.

Specifically, when the communication unit 120 or 710 receives trafficinformation related to the current location of the vehicle 1, theprocessor 170 may judge whether the congested zone 1700 is presentwithin a predetermined distance (e.g., 300 m) from the vehicle 1 orwithin an area selected by the driver based on traffic information. Thetraffic information may include information regarding various factorswhich have an effect on traffic such as, for example, the state oftraffic on particular road section, accidents, and roadwork. Forexample, the processor 170 may set a zone where the vehicle cannottravel at a prescribed speed (60 kim/h) or more to the congested zone1700.

In addition, the processor 170 may automatically enter an imageexpansion mode when the congested zone 1700 is present within thepredetermined distance (e.g., 300 m) from the vehicle 1 or within thearea selected by the driver.

FIG. 17A illustrates the top view of a four-lane road. In FIG. 17A, thevehicle 1 is about to enter the congested zone 1700 and a plurality ofother vehicles 17 to 21 is present in the congested zone 1700.

The vehicle 1 generates a main image corresponding to a first range1711. In addition, cameras mounted to another vehicle 17 capture animage of a second range 1721, cameras mounted to still another vehicle18 capture an image of a third range 1731, cameras mounted to yetanother vehicle 19 capture an image of a fourth range 1741, camerasmounted to still yet another vehicle 20 capture an image of a fifthrange 1751, and cameras mounted to still yet a further vehicle 20capture an image of a sixth range 1761.

In this way, the other vehicles 17 to 21 in the congested zone 1700 maygenerate first to sixth sub images respectively corresponding to thesecond to sixth ranges 1721, 1731, 1741, 1751 and 1761. The driverassistance apparatus 100 may transmit image request signals to the othervehicles 17 to 21, and the other vehicles 17 to 21 may transmit thefirst to sixth sub images to the driver assistance apparatus 100 inresponse to the image request signals.

FIG. 17B illustrates the screen of the AVN apparatus 400 equipped in thevehicle 1 in the state illustrated in FIG. 17A.

The processor 170 may generate a main image 1771 corresponding to thefirst range 1711. The main image 1771 may include an image 1772corresponding to the vehicle 1. In addition, the processor 170 maygenerate a new sub image 1772 by combining the first to sixth sub imagesbased on overlapping portions between the first to sixth sub images. Inthis case, the sub image 1772 may include images 1722, 1732, 1742, 1752and 1762 respectively corresponding to the other vehicles 17 to 21 inthe congested zone 1700.

The driver can visually check the real-time state of the congested zone1700 via an expanded image 1770 that shows both the main image 1771 andthe sub image 1772, which may assist the driver in, for example,reducing travel time to a destination compared to the case where onlythe main image 1771 is provided to the driver or where the driver simplyreferences traffic information.

FIGS. 18A and 18B are views explaining the operation of the driverassistance apparatus 100 according to one embodiment of the presentinvention.

In FIG. 18A, when there is no overlapping portion where a main imagecorresponding to a first range 1811 of the vehicle 1 and a sub imagecorresponding to a second range 1812 of another vehicle 22 overlap, thedriver assistance apparatus 100 may judge the positional relationshipbetween the main image and the sub image based on information acquiredby the sensing unit 760.

Specifically, the sensing unit 760 may acquire information regarding theposition of the other vehicle 22 relative to the vehicle 1 using aprescribed signal 1813 (e.g., ultrasonic waves, infrared light, orlaser).

The processor 170 may judge the distance between the vehicle 1 and theother vehicle 22, the angle θ between a center axis 1814 of the vehicle1 and a center axis 1815 of the other vehicle 22, and the positionalrelationship between the vehicle 1 and the other vehicle 22 based oninformation acquired by the sensing unit 760.

The processor 170 may change (e.g., rotate or move) at least one of themain image and the sub image based on the positional relationshipbetween the vehicle 1 and the other vehicle 22 upon the generation of anexpanded image using the main image and the sub image. For example, theprocessor 170 may rotate the sub image clockwise by the angle θ on thebasis of the main image and, thereafter, generate an expanded imageusing the rotated sub image.

FIG. 18B illustrates the state in which another vehicle is locatedoutside the sensing distance of the sensing unit 760.

In this case, the processor 170 may judge the positional relationshipbetween a main image and a sub image based on location information ofthe vehicle 1 and location information of the other vehicle 23 receivedby the communication unit 120 or 710. In addition, the processor 170 maygenerate an expanded image by combining the main image and the sub imagewith each other based on the positional relationship between the mainimage and the sub image.

In addition, upon the generation of the expanded image, the processor170 may combine the main image and the sub image with each other basedon vehicle body direction information of the vehicle 1 and vehicle bodydirection information of the other vehicle 23. The vehicle bodydirection information V1 of the vehicle 1 may be acquired by the sensingunit 760 and the vehicle body direction information V2 of the othervehicle 23 may be received by the communication unit 120 or 710. Forexample, the vehicle 1 and the other vehicle 23 may respectively includeat least one sensor which senses the direction in which the vehicle bodyfaces such as, for example, a geomagnetic sensor or a direction sensor.

Referring to FIG. 18B, the processor 170 may judge the distance anddirection from the vehicle 1 to the other vehicle 23 by comparing theGPS coordinates P11 of the vehicle 1 with the GPS coordinates P12 of theother vehicle 23. That is, the processor 170 may judge that the othervehicle 23 is located at a distance X2-X1 from the vehicle 1 along theX-axis and is located at a distance Y2-Y1 from the vehicle 1 along theY-axis.

In addition, the processor 170 may judge the angle between the centeraxis of the vehicle 1 and the center axis of the other vehicle 23 bycomparing the vehicle body direction information V1 of the vehicle 1with the vehicle body direction information V2 of the other vehicle 23.

The processor 170 may generate an expanded image that is morerepresentative of the actual state by changing at least one of a mainimage and a sub image based on the distance and direction from thevehicle 1 to the other vehicle 23 and the angle between the center axisof the vehicle 1 and the center axis of the other vehicle 23.

FIG. 19 is a view explaining the operation of the driver assistanceapparatus 100 according to one embodiment of the present invention.

Specifically, FIG. 19 illustrates the operation of the driver assistanceapparatus 100 for controlling movement of the vehicle 1 based on theexpanded image. For convenience of description, the vehicle 1 is assumedto correspond to the state illustrated in FIG. 16B. To facilitateunderstanding, the other vehicle 14 illustrated in FIG. 16B is omitted.

Referring to FIG. 19, the driver assistance apparatus 100 may generate acontrol signal that commands change of at least one of the speed anddirection of the vehicle 1 based on information regarding the obstacle62 shown in the expanded image. The information regarding the obstacle62 may include various pieces of information such as, for example, thesize of the obstacle 62 and the distance to the obstacle 62.

For example, the vehicle 1 may move from the current location P30 toenter the curved road 1602 and then sequentially pass through first tothird points P31 to P33. In this case, the processor 170 may generate acontrol signal that commands for the vehicle 1 speed reduction to afirst value or less prior to passing through the first point P31.Subsequently, the processor 170 may generate a control signal thatcommands for the vehicle 1, having passed through the first point P31,speed reduction to a second value or less, which is smaller than thefirst value, prior to passing through the second point P32.Subsequently, the processor 170 may generate a control signal thatcommands for the vehicle 1, having passed through the second point P32,speed reduction to a third value prior, which is smaller than the secondvalue, prior to passing through the third point P33.

As illustrated in FIG. 19, as the drive unit 750 of the vehicle 1previously reduces the speed of the vehicle 1 starting from a locationat which the vehicle 1 is distant from the obstacle 62 based on acontrol signal provided from the processor 170, the risk of collisionbetween the vehicle 1 and the obstacle 62 may be reduced.

FIGS. 20A and 20B are views explaining the operation of the driverassistance apparatus 100 according to one embodiment of the presentinvention.

First, FIG. 20A illustrates the top view of a parking lot 2000 intowhich the vehicle 1 has entered. Referring to FIG. 20A, three vehicles24 to 26 are located in the left side of the parking lot 2000 and threevehicles 27 to 29 are located in the right side of the parking lot 2000.For convenience of description, the entire parking lot 2000 is assumedas falling within the total range of the image capture range of thevehicle 1 and the image capture range of the six vehicles 24 to 29.

Meanwhile, a pedestrian 71 is present at the left side of the othervehicle 24. In this case, the driver of the vehicle 1 has difficulty invisually checking the pedestrian 71 who is hidden by the other vehicle24.

However, assuming that the pedestrian 71 falls within the image capturerange of the other vehicle 24, the pedestrian 71 is shown in a sub imagecorresponding to the image capture range of the other vehicle 24. Theprocessor 170 may generate an expanded image using the sub imagegenerated by the other vehicle 24.

FIG. 20B illustrates an expanded image 2010 displayed on the screen ofthe AVN apparatus 400 of the vehicle 1 when the vehicle 1 has enteredthe parking lot 2000. For convenience of description, it is assumed thatthe entire parking lot 2000 illustrated in FIG. 20A is shown in theexpanded image 2010. As illustrated, the expanded image 2010 may includea first image 2011 corresponding to the vehicle 1 and second to seventhimages 2021, 2031, 2041, 2051, 2061, and 2071 respectively correspondingto the second to seventh vehicles 24 to 29.

In addition, since the pedestrian 71 is shown in the sub image generatedby the other vehicle 24, the driver can check the pedestrian 71 hiddenby the other vehicle 24 via the expanded image 2010.

Meanwhile, the processor 170 may generate at least one route, alongwhich the vehicle 1 can travel, based on the expanded image 2010. Forexample, the processor 170 may judge whether a space having a widthgreater than the full width of the vehicle 1 is present in the parkinglot 2000 by analyzing the expanded image 2010.

Referring to FIG. 20B, the processor 170 may calculate the distance fromthe left end of the fifth image 2051 to the right end of the sixth image2061 within the expanded image 2010, change the calculated distance intoan actual distance, and compare the actual distance with the full widthof the vehicle 1. That is, the processor 170 may judge whether thevehicle 1 can enter the space between the fifth vehicle 27 and the sixthvehicle 28.

In addition, the processor 170 may detect a parking line of the parkinglot 2000 to judge that the vehicle 1 has entered the parking lot 2000 atpresent and generate information 2101 to guide the driver to park thevehicle 1 between the fifth vehicle 27 and the sixth vehicle 28.

In this way, an indicator 2101, which indicates that the parking spacebetween the fifth vehicle 27 and the sixth vehicle 28 is empty may bedisplayed on the screen of the AVN apparatus 400.

In addition, upon generation of at least one route, along which thevehicle 1 can travel, based on the expanded image 2010, the processor170 may generate a control signal that commands display of an indicator2102 that guides the generated route. For example, as illustrated, theindicator 2102 may indicate a route that guides the vehicle 1 to theparking space between the fifth vehicle 27 and the sixth vehicle 28. Atthis time, the indicators 2101 and 2102 may be displayed to be overlaidon the expanded image 2010.

In addition, the processor 170 may generate a control signal to commandthat the first image 2011 corresponding to the vehicle 1 included in theexpanded image 2010 is displayed to be distinguished from the second toseventh images 2021, 2031, 2041, 2051, 2061 and 2071. For example, theAVN apparatus 400 may display the second to seventh images 2021, 2031,2041, 2051, 2061 and 2071 more thickly than the first image 2011 basedon a control signal provided from the processor 170 as illustrated inFIG. 20B.

FIG. 21 is a view explaining the operation of the driver assistanceapparatus 100 according to one embodiment of the present invention.

The processor 170 may generate a control signal that commands theprovision of visual effects to a region, where an obstacle is present,within an expanded image. Upon display of an expanded image, the displayunit 741 of the vehicle 1, the display unit 180 of the driver assistanceapparatus 100, and/or the AVN apparatus 400 may provide visual effectsto the region, where an obstacle is present, within the expanded imagein response to a control signal provided from the processor 170. Forexample, the processor 170 may provide the AVN apparatus 400 with acontrol signal to generate effects such as, for example, coloring,styling, flickering, or highlighting, on an obstacle or in a regionclose to the obstacle within the entire expanded image.

Referring to FIG. 21, an indicator 2103 that indicates the presence ofthe pedestrian 71 may be displayed in a region, where the pedestrian 71is shown, within the expanded image 2010 illustrated in FIG. 20B. Whenthe range of the expanded image 2010 is wide, the driver may fail torecognize the pedestrian 71 shown in the expanded image 2010. In such acase, displaying the pedestrian 71 so that it is emphasized more thanthe remainder of the expanded image 2010 through the use of theindicator 2103 may assist the driver of the vehicle 1 in rapidlyrecognizing the pedestrian 71.

FIGS. 22A and 22B are views explaining the operation of the driverassistance apparatus 100 to control the display state of an expandedimage according to one embodiment of the present invention. Forconvenience of description, the case where the expanded image 2010 isdisplayed on the screen of the AVN apparatus 400 as illustrated in FIG.20 will be described below.

First, referring to FIG. 22A, the processor 170 may divide the screen ofthe AVN apparatus 400 into a plurality of sub screens S11, S12 and S13.In addition, the AVN apparatus 400 may display different pieces ofinformation on the respective sub screens S11, S12 and S13 under thecontrol of the processor 170.

For example, the AVN apparatus 400 may display the expanded image 2010on the first sub screen S11, display information regarding an obstaclesuch as, for example, the pedestrian 71 shown in the expanded image 2010on the second sub screen S12, and icons 2201 to 2205 corresponding tovarious functions on the third sub screen S13.

The information regarding the obstacle, displayed on the second subscreen S12, may include, for example, a warning image or message thatguides detection of an obstacle (e.g., “Pedestrian Sensed”), a messagethat guides the distance to the obstacle (e.g., “5 m ahead”), and amessage that guides action that the driver has to perform in order toprevent a collision with the obstacle (e.g., “Slow Down”).

In addition, the driver may execute a function corresponding to thetouched icon by touching at least one of the icons 2201 to 2205displayed on the screen of the AVN apparatus 400.

For example, the first icon 2201 may correspond to the function toexpand the expanded image 2010, the second icon 2202 may correspond tothe function to reduce the expanded image 2010, the third icon 2203 maycorrespond to the function to rotate the expanded image 2010, the fourthicon 2204 may correspond to the function to perform autonomous drivingwith respect to a route generated based on the expanded image 2010, andthe fifth icon 2205 may correspond to the function to stop the displayof the expanded image 2010 and to move to a home screen.

Next, FIG. 22B illustrates one example in which the driver assistanceapparatus 100 changes the display of an expanded image based on userinput.

In FIG. 22B, when the driver touches the third icon 2203 on the thirdsub screen S13 illustrated in FIG. 22A, the processor 170 may rotate theexpanded image 2010 by a prescribed angle. For example, whenever thedriver touches the third icon 2203 once, the processor 170 may generatea control signal that commands display of the expanded image 2010 afterrotating it clockwise by 90 degrees. FIG. 22B illustrates the state inwhich the expanded image 2010 has been rotated clockwise by 90 degreesas the third icon 2203 is touched once. When the driver touches thethird icon 2203 once again, the AVN apparatus 400 may display theexpanded image 2010 rotated by 180 degrees. Of course, it will beapparent by those skilled in the art that, when the driver touches thethird icon 2203 a total of four times, the expanded image 2010 isrotated by 360 degrees to thereby be displayed in the same manner asillustrated in FIG. 22A.

Although the above description with reference to FIGS. 8 to 22B hasfocused on the operation of the driver assistance apparatus 100 whichreceives a sub image generated by another vehicle to generate anexpanded image, the driver assistance apparatus 100 may provide theother vehicle with a main image, i.e. an image corresponding to theimage capture range of the vehicle 1. Hereinafter, the operation of thedriver assistance apparatus 100 to provide the other vehicle with themain image will be described in more detail.

FIG. 23 is a flowchart illustrating a control method of the driverassistance apparatus 100 according to one embodiment of the presentinvention.

Referring to FIG. 23, the processor 170 enters an image provision mode(S2300). In the present invention, the image provision mode means a modein which at least a portion of a main image of the vehicle 1 andinformation regarding the main image is provided to another vehicle.

The processor 170 may enter the image provision mode when apredetermined condition is satisfied.

For example, when the input unit 110 receives user input that commandsentry to the image provision mode, the processor 170 may enter the imageprovision mode. In this case, the user input may be at least oneselected from among various inputs such as, for example, touch, voice,button push, and a gesture.

In another example, the processor 170 may enter the image provision modewhen the communication unit 120 or 710 receives an image request signalfrom another vehicle.

Subsequently, the processor 170 generates a main image using one or morecameras 195 to 198 (S2305). For example, the processor 170 may turn onat least one of the cameras 195 to 198 illustrated in FIG. 2B whenentering the image provision mode, to generate a main image. That is,the main image may include at least one of a forward image, a leftwardimage, a rightward image, and a rearward image.

At this time, the main image may have various forms. In one example, themain image may be a still image or a moving image. In another example,the main image may have an around-view form as illustrated in FIG. 2C.Hereinafter, for convenience of description, the main image is assumedas being an around-view image including all of a forward image, aleftward image, a rightward image, and a rearward image of the vehicle1.

Subsequently, the processor 170 judges whether an obstacle is present inthe main image generated in Step S2305 (S2310). For example, theprocessor 170 may perform object detection for the main image of thevehicle 1, and judge, based on the object detection results, whether anyone of objects near the vehicle 1 approaches within a reference distance(e.g., 2 m) from the vehicle 1, thus carrying the risk of accidentsbeyond a reference value.

Subsequently, upon judging that the obstacle is present within the mainimage in Step S2310, the processor 170 may select another vehicle towhich the main image will be transmitted (S2315), and transmit the mainimage to the selected vehicle (S2320). At this time, the processor 170may select some of a plurality of other vehicles located near thevehicle 1 according to a predetermined reference, and transmit the mainimage to the selected vehicle(s).

Specifically, the processor 170 may select the other vehicle, to whichthe main image will be transmitted, based on the positional relationshipbetween the vehicle 1 and the obstacle. For example, when two othervehicles are traveling respectively in the front and the rear of thevehicle 1 and the obstacle present within the main image is located inthe rear of the vehicle 1, the processor 170 may provide the main imageonly to the other vehicle which is traveling in the rear of the vehicle1.

The processor 170 may select the other vehicle, to which the main imagewill be transmitted, based on the risk degree of the obstacle shown inthe main image. For example, when ten other vehicles are present nearthe vehicle 1, the processor 170 may transmit the main image to onlyfive vehicles among the ten other vehicles when the risk degree of theobstacle detected from the main image has a first value, and maytransmit the main image to all of the ten other vehicles when the riskdegree of the obstacle has a second value which is greater than thefirst value. That is, the processor 170 may increase the number of othervehicles, to which the main image will be transmitted, as the riskdegree of the obstacle detected in the main image increases. Here, therisk degree may be a value calculated based on, for example, thedistance between the vehicle 1 and the obstacle, the size of theobstacle, and the kind of the obstacle.

In addition, the processor 170 may select the other vehicle, to whichthe main image will be transmitted, based on an image request signalreceived by the communication unit 120 or 710. For example, theprocessor 170 may control the communication unit 120 or 710 so as totransmit the main image only to vehicles that sends an image requestsignal.

In this case, the communication unit 120 or 710 may directly transmitthe main image to the other vehicle. That is, the communication unit 120or 710 may directly transmit the main image to the other vehicle basedon a vehicle-to-vehicle communication network.

Alternatively, the communication unit 120 or 710 may transmit the mainimage to the other vehicle via the mediation of at least one externaldevice. For example, the driver assistance apparatus 100 may transmitthe main image to an external server, and the external server maytransmit the main image, received from the driver assistance apparatus100, to the other vehicle.

Meanwhile, although the above description of FIGS. 8 to 23 hasdistinguished the image expansion mode and the image provision mode fromeach other, this is given by way of example and the scope of the presentinvention is not limited thereto. The driver assistance apparatus 100according to the embodiment of the present invention may simultaneouslyenter the image expansion mode and the image provision mode. That is,the driver assistance apparatus 100 may provide another vehicle with amain image in the image expansion mode and may generate an expandedimage in the image provision mode. Alternatively, the driver assistanceapparatus 100 may continuously generate an expanded image or provideother vehicles with a main image while the vehicle 1 is traveling.

As is apparent from the above description, the effects of a driverassistance apparatus and a control method for the same according to thepresent invention are as follows.

Through at least one of the embodiments of the present invention, animage (i.e. an expanded image) having a wider visual field of view thanan existing around-view may be provided using an image generated bycameras mounted to a vehicle equipped with the driver assistanceapparatus and an image generated by cameras mounted to another vehicle.

In addition, through at least one of the embodiments of the presentinvention, the expanded image may be automatically generated when theenvironment around the vehicle satisfies a predetermined condition,which may improve the convenience of a driver.

In addition, through at least one of the embodiments of the presentinvention, information related to an obstacle may be provided at thetime of display of the expanded image, which may contribute to thesafety of the driver.

In addition, through at least one of the embodiments of the presentinvention, an image of an area that is invisible to the driver occupyingthe vehicle may be provided, which may assist the driver in adjustingthe traveling schedule of the vehicle.

In addition, through at least one of the embodiments of the presentinvention, the route, along which the vehicle can travel, may becalculated based on the expanded image, which may reduce the time takento reach a destination through provision of information about thereal-time road state.

Effects of the present invention should not be limited to theaforementioned effects and other not-mentioned effects will be clearlyunderstood by those skilled in the art from the claims.

The embodiments of the present invention as described above are notlimited to be implemented only via the apparatus and the method and maybe implemented via a program that realizes a function corresponding tothe configuration of each embodiment of the present invention or arecording medium in which the program is recorded. This implementationwill be easily realized by experts in the art of the present inventionfrom the above description of the embodiments.

In addition, it should be readily understood that the invention is notlimited to the embodiments described above and the accompanyingdrawings. Rather, the invention can be modified to incorporate anynumber of variations, alterations, substitutions, or equivalentarrangements not heretofore described, but which are commensurate withthe spirit and scope of the invention. Accordingly, the invention is notto be seen as limited by the foregoing description of the embodimentsand the accompanying drawings, and some or all of the embodiments may beselectively combined with one another to achieve various alterations.

What is claimed is:
 1. A driver assistance apparatus comprising: atleast one camera configured to generate a main image by capturing atleast one image of a periphery of a vehicle; a communication unitconfigured to receive a plurality of sub images generated by at leastone other vehicle; and a processor configured to: select at least one ofthe sub images based on a predetermined condition or user input; andgenerate an expanded image using the main image and the selected atleast one of the sub images.
 2. The driver assistance apparatusaccording to claim 1, wherein the processor is configured to generatethe expanded image by generating an around-view image of the vehicleusing the main image and the selected at least one of the sub images. 3.The driver assistance apparatus according to claim 1, wherein thecommunication unit is configured to: transmit an image request signal tothe at least one other vehicle; and receive the plurality of sub imagestransmitted by the at least one other vehicle in response to the imagerequest signal.
 4. The driver assistance apparatus according to claim 3,wherein the processor is configured to control the communication unit totransmit the image request signal to the at least one other vehicleshown in the main image.
 5. The driver assistance apparatus according toclaim 3, wherein the communication unit is configured to further receivelocation information of the vehicle, and wherein the processor isconfigured to: judge whether a caution zone is present within apredetermined distance from the vehicle based on the locationinformation; based on the caution zone being present within thepredetermined distance from the vehicle, identify the at least one othervehicle as being located in the caution zone; and control thecommunication unit to transmit the image request signal to the at leastone other vehicle located in the caution zone.
 6. The driver assistanceapparatus according to claim 5, wherein the processor is configured tojudge whether the caution zone is present within the predetermineddistance from the vehicle by judging whether at least one of anintersection, an uphill road, a downhill road, a crosswalk, a parkinglot, a tunnel, a narrow road, a pothole, or a curved road is presentwithin the predetermined distance from the vehicle.
 7. The driverassistance apparatus according to claim 3, wherein the communicationunit is configured to further receive location information of thevehicle and traffic information, and wherein the processor is configuredto: judge whether a congested zone is present within a predetermineddistance from the vehicle based on the location information and thetraffic information; based on the congested zone being present withinthe predetermined distance from the vehicle, identify the at least oneother vehicle as being located in the congested zone; and control thecommunication unit to transmit the image request signal to the at leastone other vehicle located in the congested zone.
 8. The driverassistance apparatus according to claim 1, wherein the processor isconfigured to select at least one of the sub images based on apredetermined condition or user input by selecting, from among theplurality of sub images, a sub image showing an obstacle.
 9. The driverassistance apparatus according to claim 1, wherein the processor isconfigured to determine whether the main image and the at least one ofthe sub images have an overlapping region in which a portion of the mainimage overlaps with a portion of the at least one of the sub images and,based on a determination that the main image and the at least one of thesub images have the overlapping region, generate the expanded image bycombining the main image and the at least one of the sub images witheach other based on the overlapping region.
 10. The driver assistanceapparatus according to claim 1, wherein the communication unit isconfigured to further receive location information of the vehicle andlocation information of the at least one other vehicle, and wherein theprocessor is configured to determine whether the main image and the atleast one of the sub images have an overlapping region in which aportion of the main image overlaps with a portion of the at least one ofthe sub images and, based on a determination that the main image and theat least one of the sub images do not have the overlapping region,generate the expanded image by combining the main image and the selectedat least one of the sub images with each other based on the locationinformation of the vehicle and the location information of the at leastone other vehicle.
 11. The driver assistance apparatus according toclaim 10, wherein the communication unit is configured to furtherreceive vehicle body direction information of the at least one othervehicle, and wherein the processor is configured to: receive vehiclebody direction information of the vehicle from a sensing unit of thevehicle; and generate the expanded image by combining the main image andthe selected at least one of the sub images with each other based on thelocation information of the vehicle, the vehicle body directioninformation of the vehicle, the location information of the at least oneother vehicle, and the vehicle body direction information of the atleast one other vehicle.
 12. The driver assistance apparatus accordingto claim 1, wherein the processor is configured to generate informationregarding an obstacle that is shown only in the selected at least one ofthe sub images based on the expanded image including the main image andthe selected at least one of the sub images.
 13. The driver assistanceapparatus according to claim 12, wherein the processor is configured togenerate a control signal to change at least one of a speed anddirection of the vehicle based on the information regarding theobstacle.
 14. The driver assistance apparatus according to claim 1,wherein the processor is configured to generate a route for the vehiclebased on the expanded image.
 15. The driver assistance apparatusaccording to claim 14, wherein the processor is configured to generate acontrol signal to enter an autonomous driving mode with respect to theroute.
 16. The driver assistance apparatus according to claim 1, furthercomprising a display unit configured to display the expanded image. 17.The driver assistance apparatus according to claim 16, wherein theprocessor is configured to divide a screen of the display unit into aplurality of sub screens, and wherein the processor is configured todisplay the expanded image on one of the sub screens and displayinformation regarding the expanded image on another one of the subscreens.
 18. The driver assistance apparatus according to claim 16,wherein the processor is configured to control the display unit toprovide a visual effect to a region within the expanded image thatincludes an obstacle.
 19. The driver assistance apparatus according toclaim 16, wherein the processor is configured to control the displayunit to rotate and display the expanded image by an angle correspondingto user input.
 20. The driver assistance apparatus according to claim16, wherein the processor is configured to control the display unit todistinguish and display an image corresponding to the vehicle from animage corresponding to the at least one other vehicle.
 21. The driverassistance apparatus according to claim 1, wherein the processor isconfigured to generate the expanded image, either in real time orperiodically.
 22. The driver assistance apparatus according to claim 21,wherein the processor is configured to change a generation period of theexpanded image according to a speed of the vehicle.
 23. The driverassistance apparatus according to claim 1, wherein the processor isconfigured to control the communication unit to transmit the main imageto the at least one other vehicle.
 24. The driver assistance apparatusaccording to claim 23, wherein the communication unit is configured toreceive an image request signal with respect to the main image from theat least one other vehicle, and wherein the processor is configured tocontrol the communication unit to transmit the main image to the atleast one other vehicle in response to the image request signal.
 25. Thedriver assistance apparatus according to claim 23, wherein the processoris configured to control the communication unit to transmit the mainimage to the at least one other vehicle based on judging that anobstacle is present in the main image.